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[y. SUMMArtY 

■ ■ -.'^ 

BACKGROUND\ 



"To assure that Federal funds for postsepondary education, 
whether allo,tted to studehts or directly to an institution, are 
used' in sDun'ti^ educational programs , they are limited almost 
entirely to accredited ins titutions * For some nursing and voca- 
tionsl schools, approval by e recognised state agency serves in 
place cf accreditation to indicate educational £f fee t44Lanas3 , 
" Because accrediting agencies ar? private voluntary associations of 
schouls or colleges which any group of institutions may establish, 
the U,S* Commissioner of Education has published a set of criteria 
that agencies must meet to demonstrate their capacity For assessing 
and making judgments about educational quality* 

" / ' 

The evaluation of an agency^^a performance with respect to the. 
published criteria is based on the agency's own self-description, 
a visit to the agency by staff of the Division of Eligibility 
and Agency Evaluation (DEAE), oossibly observation of one of the. 
agency's site vis^^Si a written repnr.t to the Commissioner's 
Advisory Committee on Accreditation and Institutional Eligiblility . 
by DEAE staff ^ and a public hearing by the Committee. The ei/alu- 
ati.ve process leads to the granting or denial of recognition to an 
agency--that iS| to acceptance or reje^ction of its decisions to 
^accredit as indicative of good educational practice,. The purpose 
of the present study was to evaluate the criteria and procedures 
on which recognition is based, e5camlnlng how well they identify 
"the agencies that can be relied on to make sound judgments of 
educational quality^. ' ■ 

PROCEDURES .1 



The DEAE staff reporte include most of the infornratiqn on 
which the recognition of accreditlrig agencies is based. All the 
reports produced by DEAE, cover'ng 234 recognition decisions from 
1968 to 1978| were analyzed for structurei consistency, and the 
qualities that distinguished recognized agencies from those denied 
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rccoqhition, in 1 977, Educational Testing Service stafr 

identified, ir^ ^ ympje oX^fLfrr r^^^^ n^ore than 500 state- 

mien to tfescTIBing agency charactferistics reieyant to the criteria 
for recognition- The most riprtsentitiv.e" of thpse were judged 
for their importence in determining, educational quality by the 
directors of about 80 percent of existing accrediting and state 
approval agencies, both recogr.lfed ^nd unrecopniied* Additionally^ 
more than 200 institutional or prografn administrators and faculty 
members, state and federal officials concerned with accrediting, 
and scholars and critics of accrediting also judged the importance 
of a similar set of statemints drawn from the staff reports* Thfese 
groups, with the agency directors, represented virtually all knowl- 
edgeable viewg of accrediting. The directors also ^mpleted 
questionnaires on (1) the effects of the recognitJon process on 
their activities, and (2) suggested changes in recogition criteria 
and procedures. 

The reliability of the recognition process was e^camined in 
relation to (1) how reliably the observations can be made that lead 
to the descriptive statements in. the DEAE staff reports, (2) how 
consistent the c^escriptise information was from one point to 
another in tfi^ recpgnitiofi process, and (?) how consistently the 
critical distinctions among agencies were maintained, from year 
to year and as different agencies were evaluated. The validity of 
the process was assessed by examining (1) how well the critical 
elements in the decisions matched ,the attributes of accrediting 
aqencififs considered by a large group of e^cperts to be most impor- 
tant irt evaluating educational quality |. (2 ) how inclusive the 
criteria are with lespect '^o attributes pertjnent to the evaluation 
□ f educational quality, and how completely the criteria have been 
applied in the recognition procaisi and/CJ) how. Well a sample of 
actual recognition deijlsions agreed with the Independent evalu- 
ations of a diverse group of experts* 

For, the comparison of independept evalystions with recQgnition 
decisions that had been made oyer the past' ten years, ETS re* 
searchers wrote composite descriptions that characterized agencies 
about which each of five type^ af decisions had been made--denial 
recognition ur recognition for periods of frorr, one to four 
ya^rs. These .composite agencies were then Judged and granted or 
dexiied recognitipn by more than 100 persons drawn from the groups 
of agency directors and others v^o had previously been involved in 
*.he study F'inally, 44 of the persons who had made those simulated 
recognition decisions met for a day and a half to discuss their 
decisions, the criteria for recognition, and the Issues to pe- 
addressed in a procedure intended to identify effective judges of 
educational programs and institutions. 



RESULTS 



Judged Importance of th e' Cril-pria ' . 

' 

*■ ' i 

! ' . ' ■ 

. The qualities of ac.credltirig agenciea judged most iniport'Jnt 
by .jgtncy directors and by the ttther groups whose views were 
solicited were those related to the development and mBintenance of 
standards, procedures for evaluating programs or institutions and 
the fairness and integrity of the accrediting process. Agreement 
on the relative importarice of agency attributes was high across all 
t'le groups providing Judgments and among directors of all types of- 
■agencies.. Those common, perceptiohs of the requirements for an 
effective agency suggest that separate sets of criteria for state 
vocational and state nursing agenejes, whi^h at present differ 
from those for priyite rsqianal and specialized agencies, aie 
, unnecessary. Differences in orqanizatlohal structure and operating 
procedjures can be accqmmodate'd readily within a single set of 
criteri^a. , 

The specific attributes 'that contributed, to judgments about 
an agency '..s per formance with respect to particular criteria varied 
wioely in judgeq importance. That an agency publishes its stan- 
dards, defines them clearly, applies them fairly, and reviews them ' 
periodicilly were highly important Consideration's in the views of 
all the groups providing judgmehts. h That an agency evalustes the 
vaiidUy-,and reliabiHty of its standards and checks them against, 
the performance of program gradubtes were seen as characteristics 
appreciabiy less important, even though they 'bear bn the same 
general concern for standards. , The importance attached to broad 
aspects bf \the criteria di£CeEed_at times from the impor tance 
attached.) to the detailed characteristics related to the broader 
issues. The discrepancies between issues as viewed generallv 
and as se^en in their eomponenfc! parts suggested a need either 
ror clarification and elaboration of the meaning of the general 
issues--such as the naturi of an 'agency's attention to standards— 
or for more searching "ways to evaluate them. 



Ju dged Observability of tKe Criteria ' 

The agency directors who provided judgments of the importance 
of various agency characteristics aJso specified those that were 
difficult to obseV.ve and evaluate. About two-thirds of the' 
qualities related t\o the criteria on which recognition decisions 
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/'?s_L. ^ert J^ydgtd to -be xaasQnabIy..-.dpef» to- obiervation* - These 
involved organizitional, structural, or prpcedural iisues, such 
as the * form s of evaluative pfoceduresi or Nfiether procedures arid 
standards are publishedi The other third—those assessed as being 
dirficult to Qbs#r-ve*-were related to the effectiveness, merit , or 
quality of sn egency's activities* Most of the criteria had some 
associated attributefe that were readily observed. The exceptions 
dealt with the fairness and Impartiality of the decisions,, adminis- 
tritive effectiveness, and finncial streng.th. Few if- any aspects 
of these qUs^lities were seen as accessible to *3bservationp The 
rtliabil it y of the observations on whleh-recdgnitlon is based 
is therefore mixedi although the criteria on the whole refer to 
atti'ibutes generally considered observable • 



Distinctions Between Agencies Recognized and Denied Recognition 

The general issues that d iscriminat&d among recognized 
agencies and those denied or given limited j^ecognit^ion were 
consistent from year to year and from one step in the recpgnition- 
process to another-^thit is, from the agencies' petitions for 
recognition to th'e DEAE staff report to the Advisory Committee*-; 
Error in the pr/Ocess was evident primarily in the detailed agency 
attributes related to a Driterion rather than in the general quali- 
ties o^f an agencyi Thus.whi|e weakness in an agency's evaluative 
procedures could be expected to be reliably observed and reported, 
the specific aspects on which that judgment rested might^.be 
repor ted . d i f feren ti y by different observers. Mora detailed 
specification of the ways agenclei might demonstrate. that they meet 
the requirements of various criteria, and agaihi clearer definition' 
and differentiation of some of the criteria, yfould probably 
improve the consistency with which relevant details of an agency's 
operations were .evaluated. With respect to the broad distinctions 
among agencies that are refiected by differences in their recogni- 
tion status, the process is quite reliable, ^ 



The agency characterlstrcs most strongly associated with 
the recognition dec3.sons were, with one exception ,\ttiose also 
judged to be most important. Recognition therefore ^indicated 
the presence of quelitles of agency organization and performance 
that were uniformly considered by highly divergent groups to be 
those most imptfrtant in making judgments about the quality of 
educational prograiTiS. The strength of their evaluative procedures 
and the fairness and Impartiality of thelr^ decisions differentiated 
strongly among agencies awarded and denied recognition and were 
also among the qualities judged highest in importance. The 
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ejcdeptlon'' to this associat ion between judged importance and 
influeh^e in the decision* dealt with the Tepreientation of the 
generar public and other donstityencles on an agency's governing 
board; while' comparatively low in judged importance, this issue .was^ 
influential in the actual decisions, 

In additloh to ahowin|b a'good match betweert Importance and in- 
. fluence in the recognition decisions ^^,the recognition process gave 
attention to virtually all agentfy ai^k^ibutes that a^a pertinent 

/ t« recognition. Nonje of the suggested^ additions to the criteria 
recBived mueti support from igency- directors is a group, nor did 
important Issues appear during th ofpSxperts that would 

indicate gapg/in the recognition W'iterla^ All but a handfulvof 
ttia j:rlteria played a role in the decisidrai although the Impdr* 
tanca dr that role varied. Jhe few criteria th^t did not affect ' 

:the dacisioni directly were those that agencies complied with 
readily, such aa havlag operated for at least. two years. The 
.recognltibn' process : is therefore valid in the sens-i that It takes 

, into account all the pertinent Issues and does so, in general, with 

^appropriate reqard 'to their 'relative importance. 



Simulated^-&fcisfon5^ \ 

V / ,^^e varied^^group of judges who made slr^Giattfd ; recognition 
dec i si ons after reviewing bomposlte descriptions of igencles 
awarded, recognition fo;r dlfferen periods agreed quite well ,wifih 

. thf;/^ctu^l decisions. ^ Disagreements centered isrgely.on agencies/ 
recognized for less ^than four\^ears* When .they dlfferedj^ thr^ 
Judges* daci lions were more lehleat than the actual decisions 

\ that had been recommended by' the Advisory Committee*v* the most i 
infrlyential ^qualities, In the^vieW^ of /the Judgesi were the fairness 
of the agehcies', decisions,! the ineffectiveness of the evaluative 

.;^proctdures*:rttjf el^^ the attention given ^to Standards, and 

thp fluplity bf vthe agencies '^^ staff^ They judged as comparatively 
ummjjtert ant tW :ifepre8entatidTi \of the general public on the 

^goy ernihg boar,d ^ the monitoring of ethical practice * The 

/ ^nsideratlona th slfnuiated decisidna thus closely 

-paralleled those found to discriminate am^^ agencies in the actual 
^decisions a^^^ v 

: x Tha conference participants ^who examined the simulated v 
decisions* and J^he existing briteria conceritirated ^elr attention on 
•.t^^^^^'X^ Their^rimary concern waiTwlBi, an ag^cy's 

^^integr'|^y--impaf^t falrneis, and independence of jud^ent* 

Th^- ieftbnrf imp^ iasue was whether thei^ivaiuafcion of what 

stadent¥ be part of 
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the accrediting phroceBs, While strong suppprt was givtnUo the 
view that good educational practict rBqulrea evaluition of the 
^graduates of programs and J.nstl tutions i some participanti were 
reluctant to lee .accradltlng agencies tike that on as a concern* 
In their; viaw, the purposes and prodiKts of educational inatitu- 

^..^^.^^ona should not be subject to sxternal review; sound procedures 
.^^^Id be the only Qoncern of accrediting. The third major issue 
invqj^red the of generality or specificity ofTthe 

recognitibn criteria and for accrediting agencies' standards, M'he 
participants felt that criteria and standards should be specific 
enough to provide usaful guides to performance but not so specific 
that flexibility and applicability arte lost. Thus, v^iile a general 
reqjj'ictmpnt, that ins tltuti gather inform'ation oh their .graduates 
was cohsic^ered desirable^ requiring that iJifotmation to take a 
rparticular \fQrm or to show'particular results would be undesirable. 
*As ^ith the earlier judgm.entS| 'agreement on the general issue was 

; broad I it narrowed as the issue bBcame more specific* . /y 

x-^ Effects of the Recoqhition PrQcess and Suggestions for Change 

^ Recent changes reported by the agency directors, have been" 
in procedQtes and sta|id not in: organiiational'^structure, 

integrity, or confcern for due process. According to the directorsj 
procedures have.becpme more, systematic i self-study 'and site visit 
procedures have been improved^ and ' standards have become stronger 
and better def^ihad . The improvement in %elf-study procedures, 
better Rrocedurfis\ for /dealing wltb^^cdff|plaints,and. greater public 
jfepresentatlon en the governing board have all been attributed to 
the recagnition process and ^ its reguiremehts . 0 the negative 
sidei the ^IrwtGrri^^ those requirements have resulted 

\^in an indreaiea.;bUtdeit!\tf^ , 

r- ; Syggf s t ion^. agreed would 
strengthens ^hejree^g^it^ focused primarily on 

ways to improve ctfrtmurii&at^ b.etwen DEAp and the agencies , 
^particularly in 'proyjjdJSig frtpre specific guiderines for meeting 
the precognition criteriy^*^ A that received a 

high level of agreement was that^ilHati^fcutigna.be held legally 
accountaJIple fqr tbe proper use tif federal funds^ None of the 
changes In^cri tftrlt that individual directors, had suggested 
attracted wide/sjjppof t. v" / - I 




CONGLUSIOiNS 



The .detailed examination of the content of the staff reports^ 
the judgments of the relative Importince of specific agency attri- 
butesi the slmulited decisions , and the cphferehce''iled' to the 
foilowing conclusidnsi the existing criteria address, the important 
, Issues in'' Identifying agencies that perfoCT effectively in evalu-. 
ating the quality of educational programs ind institutions i the^ 
criteria have been applied consistently i the decisions reached 
have been appropriate. . The study also showed, though, that while 
sub|Stantlve changes in the* criteria were not necessary, the recog- 
nitibn pfd^ess would probably be Improved through a jistructyring^ 
an^' clarifying of the; e)^istin^ criteria, and throughtdevelbpment of 
more, effective procedures for evaluating the criteria adcTfessed to 
quality of performance rather than to the e^cistence/of appropriate 
procedures. , i" , ^ . 



... , I. INTRODUtTIDN ■ = 

Billions of dollara each year flow from the federal government 
"to postsecondary education, much of it in the form of financisl 
aid to students.. To exercise some control over the value of the 
iducatio;» bought with those dollars the government requires that 
at least one of several conditions be met by educational institu- 
tions if tfiey or their students are to be Bligible to receive 
federal mone,y'; By. far the most common way for an institution 
to establish its eligijauity for federal funds— whether it is 
public ^or private, nonprofit or proprietary, degTee-granting 
or occupational--is to be accreaited .by a private voluntary 
accrediting agency or approved by a state board of approval for 
nursing or vocational programs. Yet an accrediting agency can be 
.established by any group of institutions or any other. body that 
Institutions, or programs within Ihstitutions , will accept as their 
source of accreditation. Several institutions denied accreditation 
by the aqency in their geographic region or educatUnal specialty 
could form their own agency and accredit themselvei. • To hlA««-somi 
assurance, that the accrediting and state approval agencies on which 
it relies for determining eligibility make sound Judgments about 
educational quality, t^e government has *e^st^b-lished its own 



propedurea for evaluiting tKe' agenc Thererore, for in instltu- 

J -_ / / .'- / . ■ . • ^ ' ' ' ' 

tion o^r program to.becomf eligibae for federal fundi, through 

accraditatipn or mt^e. BppvoyBlt the agency grinting iccreditation 

or approval/fnuf evaluabed- and placed on a list of 

recognized agpncles by the U.S* Commissioner of Education J 



TlT^i pT^^^ure throMgh which an agency^ gets placed on the 
Comm^sl^H^^^s aiat of recognized^agencies Is similar in several 
^/ — criteria; t^it agenciei must meet 

^^B^^^ the^^BlyHaion of Eligibinty and Agency 

illation |DEAE), the Di viil4n^ithl of Educa- 

/im (USOE) that pjovldee the supporting information for decisiong 
>ut recognition. An agency /wanting to appear on the Commls- 
lioner's list requests that recognition by submitting a petitinft to 
PEAE describing how it meets each of the published criteria. The^ 
agency is then visited by a pEAE staff member or consultant who 
gathers additional information ^ and the agency's petition is 



1- . ■ ^ ■ ^ ^ = " = ■ ^ ' ' ' 

With the establishment of the Department of Eduaatlon , the 
U*S. Office of Education and the office of U.S, Cswimissioner of 
Education ha^e been abolished. Presumably^ unless the procedures 
for aitablishlng eligibility for federal funds are changed legls- 

jlativ^ly, the proceflures. for eviluating agphcles and pubHcation 
'of ;^he ^de|*al list of recognized agencies Hiir oontlhue^ al-^ 
though^ dif,ferent/ fedetal official will have responsibility for 
promulgating ^^he list of reDognlzed^,accrediting and state approval 
agencies* Since no change had "been made' and no ^official de^ig-r 
nated to assiime responsibility for the list off recognized agencies ^ 
as of the writing of this report , reference throughout will be [ 
to Bhe U*S, Commissioner of Education. as the responsibli federal 

^fMcial^and to the U.S. Of flee of Edi^ation as^ the responsible 
fe3#ral agency. . r 



considered by a 15 = perion Advisory Commlttep on Accreditetion and 
Institutionil Eligibility appointed by the Commiisioner, Repre- 
sentatives of the agency and interested third parties may appeal 
bifore thfe CQmmittee to provide clarification or amplificifion of 
issues related to recognition. Recognition, if. granted, is for a 
period of from one to four years i At the end of that period, the 
agency may petition for renewal of recognition. The shorter 
periods of recognition are grantad agencies that have weaknesses in 
relation to thfe criteria which the Committee would like addressea 
and cdrrec£ed, or Impruved^ before a longer period of recognition 
is granted. / 

Kile the present form of the reeognltipn process was 
established in 1968, its origins are almost as old. as those of 

■ ^ ' - ^ _ ' . ' m ^ ^ - " ' - 1 

accredltinq, which goes back to-the early years of the present 

century* It was: a time ^h^en the U^S. Bureau of Edjjcatipn was 

under pressure, to .evaluate and classify colleges^and universities 

■ . - ' _ • . \ " . ^ ^ ^ ^ 

accoofddng to levels of merit * fhe more immediate origin^' of the 

current process i however, wer^ in the passage in 1^52 of a bill 

extending the GI Bill b*f World IVatf' II to Korean' War veterans. 

Congress was concerned that sorae Vetfrani might be. duped into 

MSing; their federal educational allowances In fpiyclulent or inade- 

:^pte programs j as had happened with the -first Gl Bill^^ra^mifcapg 

federal ..funds ttf ae the.- government fpmm. 

sssuranee that its funds .are being used in education or trairf|ng 

programs that arp educationally and' financially sound. Since . 



accrediting agencies are private as well as yolmtaryrthe govern- 
ment remaina outside the process of specifying' curricular or 
Inatructional form or functloni areas from which it Xs constitu- 
? tipnally exciuded. 

'i, / ' . , " ■■ ■ • / ■ ^ - V ■ : . 

Some close yarisnt of the following^ Isnguage, euthoriiing 
federal funds for higher edueatic'j, appears in more thin 20 
separi+e pieces of legislation enanted since 1952* ^ 

For purppses of this (chapt^er-s sogtion, or subsection), ^ 
the Commiss'iorier shall publish a\ list of nationally 
recognized ageneies' or associations which he determines % 
to be reliable authority as to the quility df training 
' offefeed. by an educational institution 

The criteria for determining which accrediting^^^encies to list 

were developed by USOE An consultatibn with the American Qpuncil 

on Education. These were publiahgd in the 'Federal Register of 

d^etober 4, 1952, with a list of six regional and 22 national or 

specialized agencies which were given formal ^recognition . as a 

"reliable authorjty" by the Commissioner of Educitlon. 

V ^ ■ ^ ' ■ '\ ^ . ' ^\ ^ ' \ . ' ' ' / 

In 1965 the li^t wai expanded when the Nurse fraiplng Act 

required the Cnrnmissioner of Education to publish a list of quali- 
fied state agencies that approved nut sing programs. In 1972| a 
third list was requir&d, this ane'for state agencies approving 
public postsecpndaryvbeational education. Separate criteriai mUch 
like those for poatsecondary ed|lcatioh qverall, were -developed for 
each of these groups of state' approval agehc|ies. 



Each, year, a reviitd list of recogniied accTediting aftd; etite 
i ■ ■ _ ■ ^ ■ . ' . 

ipprovel -agenciti im published, incorporating the results of the 

preceding year's decisioris by the' Commissioner, vfliieh follow 
cloaely the Advisory Committte's recommendations* With the list 
are published the current criteria for recognition, last revised in 
1974, Every agency must request renewal of its recognition and 
placement on ■ the Commissioner's list at least every four yearsi and 
must be reevaluated by the DEAE staff, and the Advisory Committee. 
The frequencies of decisibns^of differe^nt typfes oince' 1968, the 
year in which th% current eligibility procedures were :nsti£uted, 
are shown in Table 1 . Thf 234. decisions applied to more than 100_ 
different agei^cles, most of them having been evaluated more than 
once* ' ^ 

TABLE 1 

^ Frequencies of Advisory Commit itea 

.Recommendations ,1968-78 / 



Decision ^ . 


Frequency 


Percent 


Recogh^tlon for 4 years 


80 


3*. 2 


3 yeirs 


24 


10. 3 • 


2 years 


37 


.15.8 


/ . 1 year ^ : 


49 


20.9" 


Petition denied 


35 


15.0 


Show ciuse wtiy reooignition 
\ ahquld, not be reyoktcl 


8 • 


, 3.4 . 



Petition withdrawn /I 



Total /234 




.Recognition by U.sOt, like accredltation'-of an institution or 
program, is tachnlcally ^tary, but thBte art power ful incen- 

tivep for sofne/ftccredAuing .agencies to achieve recoghitionj 
Accr«.ditation/by a rtcognlzed agency makes an 'Institut^ion ' s 
students' eiig^ible for f^derai grants and loans--an anhual source 
of billions^of dollars In student aid—and mfjkes ths institution 
.eligible for gr'ants from other federal, programs o^ assistance to 
higher education. Many programfi and institutions therefore have 
felt imp^elled to seek accreditation bry recognized agencies. The 
separation of accreditationintq regldnul and specialized agencies 
» dilutes that eff set, .however, since students enrolled in collegei- 
or- un^iverai ties' accredited by a regional as-nciatlon are eligible 
fop federal assistant whether or not their particular program is 
accredited by a recognize^ specialized agency. Programs in for- 
/estry, home economics,- or chemistry, for example, need not be 
accredited by ♦■heir respective professional associations for their ■ 
students to be eligible for federal assistance if the prog ram's are 
^Hart oF^ college or university accredited by a regional associa- 
tion. Some agencies, therefore, have not .sought recognition by 
USOE. - , , ■ _ ■ ^ . - ■ , . 

^ Far -other aecridltlng agenciei, however, recognition ii 
criticjl.,, Buglness schools, for example, or schools in data , 
prdcassing^ medical techndl^y, or liw that are not attached to an 
:iccrediberf institution can most easily become eligible for federal 
is^istance bj': being aacredlt by a recognized ^ency.' Altf^ough^ 
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other ididi to eligibility are; avallabli, accredltition by a; 

i 

fecQgnized agency is the pre f erred: one, ■. ^ / 

While eligibility ^for fedlral financial aid ii the dominar^t 
inducenient for accrediting agencies to seek recognition, it is not 
the. only one. Another determining consideration iy that othir 
federal agehcies al to use the Cqmmlisloner * a list to idBhti/fy 
reputable accredited institutions. .Among them ^ for eKanple^ ire 
the Immigration Service (When admitting, foreign students), the 
Departftnt of Defense (when .approving civilian training for pepple 

on fctive duty Service Commission (ifn detRrmihing 

an applicant eligibility to sit for certain examinations). 

Recognition has also taken on somre appearance of status*/ Oust 
as accreditation is often considered by laymen to distinguish 



superior from irtTer lor institutions, so recoqnitlori is considered 



by some to be, a mark of. quality^ and recognition is sought/VwHether 
or not i4v: affects student or institutional life in any//concrete 
way. thus appreciable consequerices--bpth^ tangibl and Inwarjglble— ^ 
follow upon the recognition of accrediting agencies by t)(e Commid- 
sibner .^f Education. And the prj^esi whereb recogCltiorv and 



appearanc^rij^thB IJommlss list is either confer^d or denied 

assumes a corresponding" importance. ' 



^ The number of accrediting associations or commissions recog- 
niied through appearance on the Commissioner's ll^t grew from the 
2B/briginal agencies in 1952 to 70 in 1978, An additional tin' 



atate boards of vocationalreducafeion and eight itate boirda of 
nursing appear on separejbe iiits; During the decade fiom 1968 to 
1978, fedtral assiitanct to higher edueation through programs 
administered by the Office of Ld^ation^ which excludes veterans* 

^ :, 

educational benefiti, grtw by a factor of five, from one^half 
biLlion doll^rs^ t6 2,5 billion. The link between expenditure of 
these funds and aecreditlrig has bean to a large extent reiponsible 
for the growing scope and importance .of USOE * s process for recogni- 
zing accredit ing agencies , With that growth, concern for the 
effectiveness of the process hus increased, as well, 

THE PURPOSES OF THE STUDY 

The primary purpose of the present study was to ass^Si the 
vaLldlty and reliability of the criteria and procedures used by 
USOE^s Division of Eligibility and Agency Evaluation (DEAE) in 
det&rmlnlng v^ich accrediting agencies and state bpafds of approval 
to irecognlze/ and for how^long. There were ^ in addition, aeyeral 
related purposesi To assess the effect qf^the recognition process 
on accrediting; to -develop recommendations ' for modifying the 
rfecognlt ion process, vylth* particular attention to how the critBrla 
fqx^ rfcognition migW be weighted differently in arriving at a 
decAtioni and to review the history oT USOE's role in: r 
acGrediting agencies, with particular attention to the ""relation- 
ships and differential functions of the federal government i' state . 
MgenciiSi and ^accrediting' agencies in maintaining edLfcatlonal 



■ ■ : ■ ■ ■ \ ■ ■ ■ ■ /: . ■ ■ 

quality. Eech of thesf-parts ofHhe study wis catried out in 
relation to the purpose of recognition as de/ined in the legis- 
lative|mandat8 to the^ Commisslorier orEducatlon. The .historical 



ativel ma 
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rev lew of the criteria is Part 11 of this report 
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II. DEFINITIONS 



. . Validity and Reliability, the crlWoal cqnctpts in the evalui- 
tion of the re^cOgnltion process, have specialized meaninga which , i 
depend on contex^pHhey alsoAbowever, , uses in common 

dlsDourse* /In the^^eontext of Ihe present study, tht definltiona 
4^^-^-^^^ :^^^Mr^^nt to ;meiauFemenV.or iSEeasmentv Thoar definl^ 
tions, ho^iver, ere'olosg^to tht ^mearungs oT t^ they are 

uied in ^vepyday discourse , and they ^an bje understood in the 
cpnteKt of common., language. .\ S\ 

VALIDrTY ^ ' ' V 

. In .Qomrnon disDourse., validity refers to the degree oY truth 
inherent in a judgment or assertion. A valid ititement' is a 
true itatem'ent. While in logic a itatefnent or argument is either 
val^id or invalid, with fio Intermediate position possible, in meas- * 
^urefTient validity is vlrtually^exer. totaH-but- var«^^^ . 
almost complete validity. In its meaiurement usages validity is 
the degree ^of confidence that can be placed in an inference 
by virtue of available evidence. With respect to the recognition 
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process, validity ir the degree of con^fldence with which inferences 

• ^. ^' ^ - ^ V 

can.be driwn about the merit of acbrtdlting agericiea on the Com- 

missibner's'list as a reault of'suppo^tl^g^^B^ - /- / 

Defined as above, several important aspects of validity 
fpllow. Validity in this construction applies neither to the 
process nor to the criteria directly, but; to the inferences 
drawn from them. Some inferences ma)^ be vplld Bnd others invalid,' 
and in this context aven the valid inferences will still hold some 
degree of uncertainty/ as 
'the ability'qf an ajccrediting body to evaluate, the quality of 
educational programs, fre; inferred from^ limited observtt ions , .the 
"truthtK o^ t^jM$ inferences qmh^never be unassri i,^fal|V^but the^ 
degree of confidence they cbrhmand can be increased with" added 



suppbf ting '^vidert^/ ..^ ^ 

' -.r^i- ^^ '-^ : -^^../^.^^^^^^^^^^ ^^^^^ - 

The validity of, the: recognition 'process must refer £o Inftr^ 
ertces abput educationaa quality , since .determination of edue^tionil 
quality is thr legislatfd purpose of the Commissioner ' s 1 ist . 
Further.j the inferences drawn and their degree of validity must 
resi on some bddy^^f eyiden^ce. The process o^f validitioh, and a 

major task of the present study, Is to provide evidenw that the 

' . - , ^ ^ ' J '^-'^ ■ ^ . . 

recognition process distinguishes effective from ineffective 

>|gencies as eyaluators of ^Lcational/quarityr V - 



.Since validity is a property of the inferences fJrawn from 
a prccftss rgther than a ^roperty-of the process itself, any 
proeets^can have as many vsl idities as there 'are inrerences that 
people ch'oDse to draw, and for which evidence can be adduced. The 
val idities of primary in-terest, though, are those related to Jhe 
intended purposes o f the process , in 'this case the merifof an 
agency's jtidqments of educational qualltv. 

ft ;^3*or task in the prRseht study was to develop evidence- that 
Houlri sunport the me o'f thp recpqnibion criteria and' procedures^ in 
deter/Taninq the ability of -an agency to makt reliable judgments.' 
;abieyt;' quality of ^ducation^.; The ioferences to be validated, 
tt^ecerore, were thdse/relatea\tb judgnents of educational quality. 
if^^Congress w^nts; the Cbmrriissio bf Education to minimize the 
jne/feptrve use of federal funds intended for edi^cationai aid/^a 
val id prpeass is one ioh makes certain that educat ional funds are ' 
directed "to reputable, effective programs and institutions. Thus, 
the common and technicai uses of validity are confruent. 

The >Dst direct approach to oetermining the validity of 

process is to- ODserve Aether the. inferenbes drawn from it tan be 

suptported* The validity, of an employmerit selection process, fdr 

example, might be psseseed by comparing the job performance of 

persons selected by the'process.^h that of'persnna who had been 

re jectfirt by the process hut w?.'re; hired anyi^av, Oespite the cam- 
" . /' - '. ■ ■ r - ■ ' V - ; \ 

plexity of-definitions bf job perfQrmanee arid the variety of other 
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considerations that anter hiring practices, such as compati- 
bility with other employees, the validation of employment procssses 
is fairly straiaht forward compared with the validation of the 
reccqnition process. There are no clear performance standards for 
accrediting agencies or for judgments of educational quality 
against whici* the success of the recognition process can be 
checked. 

Ideally, validating the recognition process would take the 
form of observing and evaluating a sample, of accrediting actions 
taken by a sample of agencies. The recognized agencies would be 
observed makinq predominantly "good" Judgmfnts, while the agencies 
de/iied recognition would be observed making a higher proportion pf 
"bad" judgments t How high a proportion of "bad" judgments should 
lead to denial of lucoanition would then be another matter for 
judqment • . ^ 

No.pradtical procedure exists^ however, for observing and 
evaluating a large sample of agency decisions. Even if such a 
procedure could be carried out^ there is no strong consensus about 
the standards of educational^- quality against which the "goodness" 
of accrediting decisions could be evaluated. Evidence to support- 
or dispute the in ference that accrediting agencies on the Commis- 
sioner's list were better Judges of educational qual ity than 
those denied recognition therefore had to be indirect. 
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RELIABILITY 

^ In a context of meaaurement or asaBssment, 
IS the Cflmpara^ivB absence of error. Since some degree of' 
error is always present in Judgmental processes, reliability, like 
vaiidity, is rarely perfect. Reliability is reduced, or error 
introduced, by any discrepancy between a judgment or assertion and 
what would gentrally be accepted as factual. While validity refers 
to the confidence that can be placed in the inferences drawn from 
information presumed to be factually correct, reliability refers to 
the degree of cojrectness of the information.' 

As an example, an agency might be denied recognition partly 
on the basis that it hid exercised racial discrimination in the 
selection of members of visiting teams. If the agency in fact 
did n. Jiscriminate qn. racial grounds, there was error in that 
part of the evaluation process. The frequency with which errors 
occur over a number of evaluations determines the reliability of 
the process. Any particular decision, t^ich involves many consid- 
erations other than discriminatory practiee, might or might not bi 
an accupate judgment of the agency's ability to assess and report, 
on the educational quality of an institution. Denial of recogni- 
tion to the agency in the exmple may -therefore be correct despite 
.the error. Nevertheless, error in the determination of discrlmina- ' 
tory practice, introduces some degree of error into thr decisions , 
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and a suf Ficitntly high frequency of errors can niake the decisions 
in general too unreliable to be useful. Poor reliabilityj or error 
in the information entering decisions, always reduces validity 
because inferences cannot be better than the information on which 
they are based. Validity may be poor, howeverj even when reliabil- 
ity is high if improper or unjustified inferences are drawn from 
accurate i" formation , 

Kinds of Error 

A note on the kinds of error that reduce reliability^ in 

■ t ■ . ^ 

contrast to those that affect validity may bie useful* If an archer 

scatters his arrows falrly>broadly over a targeti the di^perklon 

^ ' \ ■ . j 

of the arrows around the target is a kind of error analogous to a 

' ; I . ■ 

reduction in reliability* The more tightly c3 usterecj the arrows 

are, the greater is the archer's reliability* Iff hbweveft the 

. ^ . ^ " - j 

archer, mistakenlv shoots at an adjacent target not assigned to 

him, that error would be analogous to an error In validi'ty, no 

matter how tightly clustered the arrows were in the wrong i:arget* 

Every arrow in the bull's-eye of the wrong target adds;to the 

reliability of an invalid performance. Error that affects relia- 

bility is associated with inconsistency* Inconsiatehc|y in the 

recognition process thus Indicates reduced reliability regardless 

of what the true suate of affairs may be. , ■ 

/ • ^ 

Many of the errors in the' recognition process are |'dif ficult v 

-■ . ■ ■ ^ ^ , 

' ■ ■ ' ■ /;■■.■■ ■/ 
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to Qbserve directly for a variety of reasons. Althdugh it is true 
that some factual errors in DEAE reports prepared for the Advisory 
Committee are corrected by agency staff at the Advisory Committee's 
meeting, other kinds of errors are not easily amenable to correc- 
tion* These errors are the ones that arise from agencies* inter- 
pretations of the kinds of infoririation pertinent to each crite^rion, 
DEAE staRf members* interpretations of the information they were 
given p Advisory Committee members' interpretations of the reports 
of DEAE staff members, and interpretations by all these groups of 
the meaning of the various criteria* 

When assertions or interpretations cannot be checked against 
concrete facts, the degree of error is estimated by comparing 
independent perceptions of the same object* When there is 
variation /or inconsistency in such perceptions, error is assumed 
wlthogt^reyard to which perception might be the "true" one* In the 
.present study, therefore ^ determinations of rellabiHty were /based 
on various observations of consistency—between the judgments of 
different kinds of judges, different observations p and different 
occasions on which the judgments were made* 



in, GENERAL APPROACH TO THE STUDY 

An iiTimediaLe problem in planning the study was the spscifica- 
tion of what was to be evaluated. The publishsd critiria and 
DEAE's procedurts in implemBnt.ing them, which led to racognltion 
decisions, were the entities be e-. alusted, but these are 
difficult to observe except superficially. The criteria, while 
published and available to every accrediting agency and to anyone 
interested in reading thtm, don't in themselves describe how 
decisions are reached, Everi the judgments of a large panel of 
experts evaluating the criteria would not have been informa- 
tive, lines extensive expert judgments over a period of years had 

already entered into the formulation of the criteria. New judg-. 

ments-. drawn from the same process could not be expected to provide . 

much information that would be either new or useful, 

jpirect observatloha of DEAE's procedures would ba far too 
coitly if carried out^by eystemitleally observing the staff members 
who ^eviewtd petitions for reeognitlon, visited «genclee, observed 
iting teams, and prepared reports. Nor could observ;atlon& be 
ids of :tha recognition activities of prior years. And without - 
some organizing set of . prlnclplas to provide m interpretive frme- 
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work for. such observ at ions , they would not be useful enough to 
justify their cost. 

One reviewer of an early questionnaire , objecting to its 
elaborateness, asked' why accrediting agency diractore and other 
knowledgeable people couldn't simply be asked in a straight- 
forward manner how good the criteria and the resulting 'recognitdon 
decisions were* One answer is that the recognition process is too 
complex for broad judgments about it to carry much meaning* A 
Judgmeht that the process was generally good would then have to be 
elaborated to specify which parts were good, which not so. good, in 
what contexts, and for what reasons. In effect, that is what was 
done, although the process was eliborated into selected details, 
and tha solicitation of judgments about their valy? was cirried out. 
systematically* ' - - 

SPECIFICATION OF THE PRQCESS ^ 

The recognition process starts with an agency submitting a 
petition for recognition stating how it has canplied with each of 
the published criteria,^ Customarily, the petition is clarified and 
expanded through correspondence, through observation of the deci-^ 
sion-making body, and perhaps through the^^se^*ation of a visiting 
team, All^iht information available is then organized by staff 
members of the Division af ..Eligibility and Agency Evaluation. (DEAE) 
into a report that details how the petitioning agency meats or 
fails to meet each criterion. 
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^ Staff reports are accompaRied by a summary which highlights 
the major lesues described in the report. The Cornmieeioner ' s 
Advisory Committee reviews the DEAE staff report, llstena to a 
preaentation by a representative of the agtncy who clarifies and 
elaboratee on questionable points, and sends a written recofnmenda- 
tion For or against recognition and a brief statement in support of 
its position to. the U.S. Commissioner of Education. Since the 
staff repp rts. provided the most detailed information on the nature 
of the discriminations that enter into the recognition decisions, 
they were made th'e basis for much of the evaluation, and expert 
jL&gments about the recognition process were^irected primarily to 
th4 de tailed contents of "the reports. The stud^ also drew, on the', 
staff summaries, the Committee's recommendations, and a sample of 
the initial petitions to fill out an extensive, detailed descrlp-^ 
tion of the recoa^nition process and tlie roles of the various 
criteria in it, 

^ During the 11 years frorn 1968 to 1978, 234 separate recogni- 
tion decisions were made, involving 114 postsecondary accrediting 
and state approval agencies of various types, as shown in Table 2r 
Each agency was evaluated from one to five times during :th'at 
period, with 29 of them evaluated three times or more. Early in 
the study, a representative sample of 52 of the staff rapbrte .that 
preceded the decisions were read, and each descriptive statement 
made about the agency, was listed with the recognition crit#rlon ' to 
which it referred. The statements were reWDrded when pecessi¥y 
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TABLE 2 

Characteristics of Accrediting and 
State Apprpval Agencies on which 
Actidns Were Taken, 1968-78 ' 



" Characteristics Frequency Percent 

Type of Agency 

Regional 20 8*5 

National institutional 40 ^ 17.1 

Specialized 118 50.4 

State vocational 35 15.0 

State nursing 21 9.0 



234 10Q.O 



Jfars of Operation 



Less than 5 years 59 25* 2 

5-9 years 32 13,7 

10-20 years 21 9.0 

More than 20 years 94 40.2 

Unknown 28 12.0 



23^ ^ 100.1 



F ield of Agency 



Medical, Dental, or Allied 

Health 100 42.7 

Ot^er 134 . 57,3 



234 100.0 



Predominant Educational Level 



Leps than 4-year post- 
secondary program 75 32. t 
Baccalaureate 7 /3.0 
Graduate 45 19.2 
Mixed J07 45.7 

N - . 234 100.0 



to make them applicable in principle to any .'^gency, and statements 
that differed only In minor details were merged into a single 
qeneral statement- A total of 559 statements resulted, with from 
10 to 130 appearing in any single staff report* The more specific 
of the criteria had only 5 to 7 statements asaoelated with themi 
the more complex had 25 to 27. Criterion bZiiB, for example, 
states, ^jiXhe agency or issociation publishes or otherwise makes 
publicly available the procedures utilized in arriving at decisions 
regarding the accteditation status of an institution or program," 
This fairly specific criterion /was elaborated in the stiff reports 
with one or more of only five separate itatements. One such 
statement is, "The proceedings of the agency's policy/decision- 
making body are published," ^ " 

In contrast, Criterion b8 reads, "It (the agency or associ- 
ation) accredits only those institutions or programs which meet its 
published standards, and demonstrc s thiTt its standards, policies, 
and procedures are fai^rly applied and that its evaluations are 
conducted and decisions rendered under cohditions that assure an' 
impartial and objective judgment*" This criterion required 27 
different statements to describe the various pertinent agency 
eharacteristics. ^ An Illustrative statement is, "Clearly estab"^ 
llshfed procedures ensure impartial evaluations and Judgments," 

Of the 44 criteria, a count that includes the subsections ' 
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within the more r^nsral criteria, 23 had fewer than 10 descriptive 
Stat ements issociated with them. Five had more than 2D, 

I While the statements culled from the sample of 52 staff 
reports describe the agencies in relation to each criterion for 
recQgnitionj tiis statements are not the criteria themselves. 
Some, for example, describe the organizatianal structura and 
purpose of the agency, the number of programs or institutions it 
has accredited, the length of time it has been operating, snd its 
relationship, if any, to other orginizations , They may b.e simply 
descriptive of some aspect of the agency related to a particular 
criterion, with no evaluative implications. Others, while obvi- 
ously evaluative, may refer to only a small portion of a criterion. 
One such statement, which refers to only a small part^of the 
criteria related to due process, is, "The agency's written pro- 
cedures for initial applications for accreditation are not clear*" 

The 44 criteria were thus elaborated into 559 statements that 
represented the applicatlcn of the criteria. Several procedures, 
described in Chapter IV, were used to -^roup the statements and 
criteria into broad sets of related criterli. The groups were 
eubdivlded int^^^he crltefia, wf-iich in turh were elaborated into 
the 559 statements* This organization of the criteria permitted 
expert judginents to be directed . to selected aspects of agencies 
that represented the full rarige of information on which recognition 
decliions were based, . ^ 



JUDGMES^TS ■ . 

The ultimate authority for any determination of validity, 
_ quality, or rrverit lies in informed judgmentb. Even when an 
Gbjecti've indicator of fnerit is possible, such as volLme of sales 
in evaluating the performance of stockbrokers, expert judgment is 
rtquired to identify and define that criterion, to determine its^ 
importance in relation to other aspects of performance of to the 
circumstances of that performance, and in general, to givi it 
meaninq within a total context of stockbroker behavior and produc- 
tivity, ^ The absence of any loceptable objective indicator of the 
ability o f an agency to assesi the Educational quality of an 
institution was not^, therefore, a genuine hindrance to. validation 
of the process,' It only meant that the ipprojach would be to 
informed Judgment directly ralhar than«throu5h interyening indi- 
cators, ^ ^ . 

The quality of judgments can vary appreciably, depending on. 
who the judges, are, what is being : judged , and how the judgments 
are made. To ensure that the Ji^gments have value, two types of 
procedures related to the quality.pf judgments can^e formulated, 
On^ type' involves the selection of the judges. They must be 
knowledgeable about the objectr^being judgedi there must^b^? 
enough of them for their cdllectlve judgments to override any 
peculiarities oF partleular individuals or muaual droups of Indi- 
viduils; ^thry should be diverse wough to repreaenL the cange of 
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perspectives from n.iich the objects Judged are likely to be viewed. 

The second type of procedure for useful judgments requires 
Clear specification of the □bjects to be judged and of the nature 
of the judqments tp be mady. For example, judgments about an 
entity- as broad and diverse as an undergraudate college must 
themselves be broad. Even though some judgments about institutions 
as a rfhole are necessary and useful, they can often be improved by 
limiting their SDOpe* If the purpose of the judgments about t\m 
college is to evaluate its graduates for admission to graduate 
programs in political science, a more precise specification of the 
object to be judged will produce more accurata and useful judg- 
ments * The judgments could well be limited to the political 
science department rather than the college as a wholes wd^perhaps 
to the upperd iv isidn political science courses or to the faculty 
members teaching upper-division political science. Excluding the 
effect of extraneous gspects of the college from judgments about 
its political science graduates increases their clarjity and 
accuracy* / 

Spfcifyiny the nature of^the judgments to be made further 
increases their usefulness. A global Judgment about the ijuality of 
upper-*division apolitical science courses at an institution is not 
^s informative as a group of Judgments directed to particular con- 
tributors to the overall quality*. Judgments about the d^ree to 
whicH quantitative methods are Hram on in analysis, or the extent 



to^ which political philoso^y is incorporated into other areas of 
poiitical scieare, for example , will clarify the meaning of any 
broad judgment of quality. This kind of specification of the" 
nature of the judgments to be made, a? well as of the object to be 
judged, improves the judgments in two -ways. The judges are more 
accurate in their judgments^ becai/se of the narrowed scope of. the 
Judqments they ate \f ked tc^ make, and the intf rpretatiaris of the 
judgments are clearer for the same reason. 

Most (3f the informed judgments in the present study were 
directed to samples of ststements that represented the criteria^ 
and groups of Driteria. The judgments about the importance of 
specific statements could then be translated ijito judgments about 
the criteria. In making theirf judgm.ents ^ the judges considered 
the ability of an agency to assess and repoft on the educational 
quality of an institution or progran. The judgments were. therefore 
limited to educational quality butAere not focused more precisely 
than that becsuse thrr rricognit/on process iteelf is not more 
sharply defined* The disiirslty of judgments that appear with ^ 
respect to a broa Jly general char a.^ter istic is itself useful 
information in^aluatiog the mtflt ,;f that characteristic, 
Judgments of merit cannot be expected to be mnre consistent than 
the de rif- A >: ions of merit, 

Tne judges included executive directors of iccrediting and„^ 
a i^te approval agencleSf administrators and faculty members in 



educaLionai inst i tutions i stqte .snd federal of ficisis concerned 
with some aspect of accredflatiQnf end admixed group of scholars 
- and,.cri tics o f^^^^a not associattd with one of .the other 

^ groupo , «Soma groups , such as the sgency dlractsrst included all 
identi f labia members &f *:he-groi^. Every director of every identi- .v 
fiible accrediting or state approval isgency in the country, whether 
appearif^g on the Gommissioner^s list or not, Mm invited to provide 
^judgments about the recognltJon process* Other groups were^^not ' 
repr^eentstfve samples but were chosen to provide diversity of 
viewpoint the faculty members and administrators j for exi^nple,' 
were selected from public and private colleges and universities, 
iarg^ and sipallj and from proprietary schools offering programs at 
all poet^^f^pcondary levels* ' , 

V: In' addititsn to the main^^pi^'^^^ of the study, several related 
go'els wrri? pursued, sufrh ai analyses^ of the Tecagnition decisions 
ind o'f their relationship to various aspects of thfc agencies, 
interviews with directors of accrediting agencies, and surveys of 
thfe ;directeKrs designed to ei-lcit their suggestions about/ how the 
recognition prdceas might be improved%- The specific procedures 
arid their res^ul^s will be described in the remainder of this- 



IV.*' THE STRUCTURE OF THE CRITERIA 

The current publiihed criteria for recognition of iccrediting 
and state approval agencies by the U.S. CQmmlssioner of Education 
(Appendix A) bav^ Four, sections which call for the evaluation of 
an agency fUhctional isptcta, reaponslbilitl, reliability ,^d 
autononiy * \ 

F^unctional aipects refer to the agency's sco|pe of optrations, 
its staffing, and prqanizatibnal and financial structure, and its 
aeere'diting procedures and each of^theie segments i? futther 
elaborated Into as many as five more specific criteria* 

^ ^ Raiponsibil ity refers to , the agency's responsiveness ' to 
the needs of its several ponstituenclei and the' general public, 
and; is the most ex ten^iVe 0 f the sections, with approximately 30 
specific requirements . These^ include requirements for making^ 
information about the igency pubricly ayalliblei publishirq the 
procedures it follows in. evaluating , accrediting r and withdrawing 
-accreditation I providi^ . a regular process for programs or insti- 
tutions to appeal adverse dacislonsi and developing effective 
stagdardSt maintaining them, and.^applyi^ them'fairly* ' 



V Rel iabllity includes four spicific criteria-*thg igency's 
proressional reputation and abceptanci, the prde|'lineiS >nd cm- 
struetiviness of its procedures, itsvascperieftea, and the inclusion, 
in its affairs of reprtsentatives of its constituencies* 

Autoromy^p including only tv^ specific criteria. p is concerned 
with the agency's e^trcise of incspendent judgmint and freedom from 
conflicts of interests. 

Somi degree of relationship among the criteria is required by 
their common bearing on educatioHaL quality. They necessarily 
overlap* One of the criterlA iinder Tesponsibilityi for exanplfe* 
requires that aqenoies "take into account the* * , interesta^^pf .* *the 
academlCj profeislonal , or occupational fields involved*" -Another, 
critaribnp this one under reliability, calls for, agencies, to 
reflect, in the composition of their policy and decision-making 
bodies I *'the community of interests directly affected by the scope 
of its acdreditation. " While a governing board that includes 
representatives from all the constituencies affected by its' actions 
does not necessarily assure that all pertinent intereatjB are taken^ 
into account, the two criteria are clearly related* Qualities that 
satisfy the criterion on .reliability inevitably have an important 
effect, no doubt appropriately , on the agency's reiponsibility • 
Th^ relat ionships among the criteria clarify their meaning , 
pointing out their distinctions as well as their areis of con* 
vergenci. A preliminary task of the study was therefore to 
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identify thr,o.perational relationBhipa among thr'crit^Via— their 
meanings. as they 



THE .DESCRIPTIVE 



STATEMENTS 



A totil of 



are applied in"^eyaluati'ng agtn^/ei,' 



559 statemB^ta descpaptive o.f aqenc^ charactBr- 
istics, elaborationa of ths^li pubLfshed critsria, weTjk cataioguea 
after several , persons read's/and pferead a random sample' of 52 staff 
reports. Thesf statements ,'wh^le not synonymoua with Che criteria, 
included m^it of the obseif,i/ationa about agencies andlthe basea on 



v3hich the/y were Judqed^o aatisfy or fall to aatlsfyj the criteria 
for recognition. The^' agency 'characteristics not caitalogued were 



those 'that occurre^d^too rarely to be Included In a Iset of atate- 
ments dealgned for^general applicationi and thoae tha^ were brought 
up in the Advi:aery Committee meeting by agency rripresentatiyis 
or others ffter the report oh the agency under review had been 
written./ The rare statements, while possibly criticilly Important 
in a particular decision, did not affect the deciaion process 
in general. When the issues raised belatedly, ai: the Advisory 
Committee's meeting, were pertinent to more . than a handful of 
.decisions, they could be dependsd on to appear-lri other staff 
reports. ■ - , ■ 

The statements' that contributed in a systematic way to the v 
decisions were those that appeared In the staff ieporta more 
often than occasionally and leas pften than all thi time. Even 



though tha^ 559 statements were forrr^uritad to apply generally , many 
ippeared In only a handful of reports, and a few occurred so often 
that they pimply diS not discrlm^inate among agencies. The first 
criterion, for example , requirea that agenciei be national or 
MgiQfialMn geogrspfiic scope. The statement that the agency's 
national or regional scope is clearly defined appeared in 89^per- 
cent of the reports, and thfrefore rarely discriminated among 

- ^ . *r ' . ■..^.L..._.. - ^ - . ^ ^ ^ 

agencies t ^^hly 1 66 ^statements appearea^ih more ^Han 10 percent 
and fewer than 90 percent of the "reports.. These 166 statements 
described much of the variatiop among agenQiei associated with 

r.:ecognit ion decisions, and the total group of 559 probably 

' ' ' ■ - ■ * ^ ■ ^ .. 

addressed almost all the fpertlnent considerations. 



Factor Analysis of the Moit Dlscriminatinq Stgtements ^ ^ 

When several statements consistently appearfed tog.ether in 
descriptions of agencies, and rarely ocourred if, others in the 
group were not also present, those statements could reasonably be 
assumed to represent collectively an aspect of the^agencles that 
was more general, than the statements themselves* .Factor 'analysis 
is a procedure through which all the relationships among the. 
statements, or imong preselected subsets of them, can be examined 
s>;s tematlcally to Identify those groups that, through . their 
tendency to occur together, indicate general aspects of'the 
agencies. Thus several stittm,ents reftrrlng to various aspects 



of an igency's financial situation, if they tended to occur Jointly 
among the total group of agencies, could be merged Into a general 
conclusion, about the agency's financiil strength. Further, the 
relationships among those statementa in a number of staff reports 
would indicate thae financial s;trength was a general quality of 
agencies that entered into the recognition dacjaians.. The groups 
of mutually related statements, or factors, Unaioate the structure 
of the criteria— the distinctions .among thejj and .the areas of 
overlap— as they are applied in th e rocognition prgcess. — 



• r|t a factor analysis to produce reliable results, the numb e^^"- 
of casei (staff reports) in which the variables (statements) ^jT^bI 



ractored a.re observed"should be four or five times the n^ber of; 
vaHabaes. With 234 staff reports, ebout 5Q-statemen#(ban bS 
reliabay factored. The 166 statements that showed a-d^east'a^ 
moderate degree of variBbllity were therefore formed iito six 
qyerlapfjing groups of from 40 a 50 variables each. The grbupS 
were foriried simply on the basis of judgment as to the kinBS^ of 
Statementa likely to be related to one artother," or for which/the 
distinctions , were unclear. Statements . concerned with ethical . 
practice-, due procffss, and a.utonomy were Included in one group,. 
Responsiveness to various constituenclea and autonomy werV part 
of another group. ' . - * 

Because of the overlappipg content of the six groups, similar 
factors appeared In more than one group. Those results, without 
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duplicited faetorif >aff summarized in; Table 3, Since the factor 
dfscrydbed in Table 3 were^ derived frwn stveral separate anilysei 
/oF different groups of stateintntsi they are preaented individually, 
Tht table g ives .the identify ing number and'^content df^the state- 
ments i the number:, of the criterion to/which each atatement 
referred, epproxlmata factor loadihga (since the factor loidings 
varied slaghtly from one^^nalyila to another), and a descriptive 
title for each' factor* The factor loadings are correlation co- 



efficients between the gtatemefits and the factor. For e^OTple, 
Statement 370 shows a correlation of ,*85 with the general quality 
of openness or responsiy^ess represented by the group of state*, 
raents labeled Factor A* * 

TABLE ^ 

Factors of Criterion-Related Statements 

A. Openness to Public Viewi Reaponsiveness to the Conberns of 
Interested Grpupa 

, Related V . . Approximate 

Criterion. Statement Factor Loading* 



b2liE*f 370 Description of orginizational *85 
y control is published. - ^ 



bZiiO 360 Affiliations of board or *80 
/ c commission menbers are ^ / 

;^ published . / 



/ 



c4 700 All constituencies are *80. 

represented on the board v 
or^contmisilon. 



■/• ■ 
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lABLE 3 . . ■ 
' ■., s ■ - ••(cont'd), 

Opennesa to PubHc Vlewi Responsiveness td the Concerns of 
Interested Groups (cont'd) ,n ' ' 

Reasted ■ j Approximite 

Crittraon Statement ^ Factor Loading 

bZiiD - ^69 Tht namte of tht principal , *75 
idmlnlatratdra are published. 

bio 650 Instructions art provided to ^" ^^ .70 
eneura that references to 
^Mredltajtlofi are accurate* 



bit, ; 280 The Interests of ^all const Itu- ;70^ 
encies are considered . 

b21 __J_83 The public Is represented on the ^ .60 

' ^ board or eommisalbn* j ' 

bZllB. 340 the nature of the decision-making ,55 

- process, is published*. 

bZili 380 Stendarda are not /revised until ' ,50 

■ interei ted grougs have commented* 



B.^ Due Process 



a2v 1^0 VABiting' teams Include at/ ^ ,75 

Y . l^ast one: person . not on 'the 

, . agency staff or commiislon. 



^vlli 490 AccWitation cannot be *75 



^,J^'^' ^ k channed .during^ an appeal, 

i^/ . 3m W^: ieten prpcedUrM provide 

r fair treatment of complaints* 



.70 



bSvJli peasons are given for adversa ,70 

^ecisiona on 'appeals* 



^b2iil 3ao Standards are not reviaed until ,65 
Vintertstad groins have commented* 
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TABLE 3 



B, Due Process tcont'd) 

Related ^ ' ^ 
Crittrlon 

/ b3vil 470 
... . b3vl 460 



Statement 



Reaesns are given for adverse 
aeoreditlng decisions « 

Acereditation is withdrawn 
^ only after du e-proc ess. . ^ 



Approximate. 
Factor Loading 

.65 



b3viil 471 
b3viil SOD 



bli 



62 



280 



64 



Appeal procedures* are publlshad. 

Adverse decisions are. fallowed 
by a hearing* 

The interests, of ell oonatltu- 
encies are considered 

■ ■ ■ - * 

Written policies and procedures 
guard igiinst ocnfllcts of 
intirest. 



,60 

.60 
.55' 

. .50 



C. Fiir PTaetice 
b8 590 



bS 



bS 



d2 



b7 



591 
■411 
M 



"Accreditation' proce'durss are 
fairly applied. 

Decisions are fair ind consist- 
ent with standsrds. . 

Standards are clear enough 
,to iiaure fair treatment. 

Written policies and procedures 
guard against conflicts ^f 
interest. ? / 



582 Programs are free to use a 
• s variety of educational rnethods. 



bli 280 The interests of all conatitu-i 
encies are considered. ' 



,75 



,75 



.55 



.45 



i4Q 



.40 
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TABLE 3 



C. fqir. Practice (Wit'd) . ^ . 

\, _Related ^ \ ^; ApprQ^lmate 

Cnttrlon \ , Statement Factor Loadin 

bSviii 510 Reaionk are given for adverse .35 
. decisions on ap^Lals, \ 



d1 



710 No fuhbtidn of the agency Inter- 
— fares with Vndependertt judgmenti ,31 

b3vii 470 Reasons are given for adverse .35 
/ ^ accrediting decisions V 

b2iii 380 Stsndardi are not revised until .3.5 
interested groupi have coinmentad, 

b8 ,610 Doubt axi^'ta about the imparti- -.35 

ijlity and objectivity of the ' 
^ decisions. ' ^ ye 

D* Evaluatipn Procedures 

V * ^ b3i , tflO The first ayaluation is at tha ^75 
* request off the institution' s 

:\ / * ' chief eKecutive officer* - ^ 

V ■ ; ' . ■ . . ■ . ■ ■. • -^..^ , ■ ■ ' 

b3v 450 At least one .Visiting team; mem .70 
\ is , present at the evaluation of . 

/ ' tha^jDeport * V ^ " 

■ ■ ■ ■ ■ - ■ . "■ 

b3iv 441 The 'Inatitution's chief e^cacutiva ,60 
officer has an^pdrtuniby to\^ 
respond^j:a-jtha: report » - - 

b3ii 272 Visiting taamS meet with faculty, .60 
administrators, and etudenta. 

bJiii^ 261 Visiting teams provide # / " ' ,55 
^ r writ tin report^of strengths ^ ^ 

and weaknesses* ^ ' 




\ ^ • TABLE '3 

I ' (cont'd)v^ 

Eyaluatlon Procedures (cant' d) . , - 

. ■\.' , - ' ^ V ' ^ ^ / ^\ '. ' ^ ■ ■ 

Related- . ' \ Appr.Q^cimat 

Critetlon ' / Statement ; Factbr Laid 

b9 ys, 54, Inatitytiens or prdgram^ are " .45 
y ■ , \ regularly reviewed at reasonable^ 

intirvsls. . 



a31ii .128 Written InstrLctlons arte pro-. #40 
vlded for the self-study and 
on-^site visits* - ^ 



Evaluation Procedural Self-stydy * 

\ a31ii 240 A /self«study and on-site visit .70 
.sfte requirad* ■ * , 

, aJiil 253 Self-study procedures are not **60- 
* , suFFiolently crltioal or 

analytic* , ^ _ 

a3ili 128 Written Instructions are provided .50 
'for the self-study and on-alte 
, visit ' ^ • 

b6 570 Regu^^ar reports are required *50 

\ - Ifidacating continuing^ 

evaluation. ' ^ 

a311i 237 The seif-study involves all . ^ •45 
* ^ constltMencieSt . ' 



Evalustiori Proceduresi Maintenance of standards and 

' protection of institutions' 
■\ interests* r' 

\ bS 599 Accreditition is withdriVi when ,75 

standards are no .1^ niet * 

aJlii 243 Visiting teams examine qualitative .55 
aspects of an institution or 
r ; program. , " 



TABLE 3 



CvaluatiQn Prooedureii MMinttnance of stindarda and 

protectlpri of inatitutions' i 
iriterista (cont'd)/ - 

Ralattd ^ : ^ % ^ Approximatt 

Criterion Statement Factor Loading 

.aZiv 124 Inatltutioni or progrma may ^ .45 
rTCommend and ,#eject^visitlngr 
/team membsrai ^ ' . 

' d1 743 Board or cojnmiaalon membtfa do^ . ,45 

not :partle;ipalt in deciaions , % ^ 

about thiir own institutions* * 

» x'^.. , . ^ / 

b3viii 500 Adverae deelalona, mtm {Vllo^pd *40 I 

' .• ' • by a hearing* ; . if ■ - x, 

* b7^ |82 Program8r art free to _ ,40 

variety of tducatlani^J^ 

a2iv 129 Visiting taams aM tr^ ,40 

b9 598 inat^utiona or programs m% ,35 

! br evaluatad at intarvila 

^ ' shorter than the normal cycle t " 

-. - ' ^ ^ . ' ■ - ^ . ^ / ■ ■ ^ #^ / - 

b3vi 460 Accreditation^ la withdraw only ,35 
aftrir due process ^ 

b3ii 421 The obaeryed vlaitirg' teim\met .35 , 
'With all appropriate groypa. 

Org^lMtional, Strength . ' 

a2i 50 The igendy is adequately staffed. .85 

/ ^ . ■■ ' . . ""^v, ' ' ^ • " • ■ - > 

a21 51 Prbcedures' ar e timely and ..^.70 

; ' effective^ " , - - 

a21v 170 Competent I qualified people ^.45 
are,ofi the ^olldy or decisioh- 
' making body; * 



•5^ 
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■ ^--^^ TABLE 3 " 

Hm .finineitl Raeponslbility 

Related;^ % . ^ Appw^lniate 

Criterion, Statemartt Faetor LQadihg 

a211 71 Fiical^ audilis: are performed 
^ regularly and flnsnelal 

■ -T- — — atatfminti are published,"— ^ 




^a2ii - / 72 FinanQial rasourcea are adequate. #40 
aZidi 100 Fees are reisohablet V #35 



I* Pfaaccreditation , V 

' : aJii ^ 220 Prpctdures are provided ^r #85. 

: ' ^ ^ . the move" From. preaQortditation 

. V .to iccreditatibn# 

a3ii 235 .Preac^reditatipn la appropriately #70 
J \ related to accredited status* 

^ a31i- ^ 225 Preaccraditatltsn ra^lraa a self- ,55 

n " ' : studfy and site visit* ^ , 

\ > ^ a3ii 231 Preacereditation is ^r '^^limited ,45 
\ period, , - 

a3i4 206 The agency ^doea not have a pre- *#7Q ; 
^ accreditation status, ^ r 

*li!ie factor loadings can be interpreted fs correlation 
coefficients between the statements and the factors, 

♦^The Initial letters in the crlterilm designations place the 
. criteria in Wie following groupsr ^ 



' a* f urtctiOTal aspert^ , 

b,: Responsibility ^ ^ . v 

^ c,; RftliabiUtx- ' 

d# Autonomy, / 

The order of the factera is an approximate indication of their 



CQiitfibution to ^the discrimination wiong agencies in the DCAE staff 
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rtports. The agincy attributes aaaoclated with FactorjA, which 
^ is eonearned with ppennasa to public view and responaivenMe to 
inte^estffff groups , ,cQllectivt^y cotf^trlbuted faf more to th 
dlocriminetlon among aganciea than did those *asaociaterf wilth 
, Factor G, wh;ich is reiatad to organiMtional strength* This does . - 
not mean that opennMs or- rasporiaiveneas is givaa more einphasla 
than ^rganiiatibnal strength in the recognition processV But v - 

Ieneiea vary more with rAapeot to openness or responsivSnass, 
cording to the, DEAE reports, than they do with respect to organi- ' ' 
/national fsfrength. -Organiiationai strength la therefore less /- 
prominent as a broad discriminating quality than is opennes^s/ 
^spohsivwasS. ' ^ ' . 

. Factor A is associated prlmfrily with stat^ents describing ^ f 

. an agency U publication of inrormation about its organizational ^-^^ 

structure, Cpmpllance with thi statementr is readily accomplished 

an^ readily observed: Ambrig^^ The statinents that; \ 

contribute to Factor 8 are related more to prcfcedures, safeguards 

/ for the accredited institutions or programs, than to OTganlMti^rials ^ ' 

structure, but they are alio readily eompiied Vlth and readily V - 
- observed- 

Factor C is associated wi th sfcatecnents: conqtrning ;fair 
.^Ractice, .Soma overlap with Factor B on due process is to' be ■ 
. m%pe^tp^^.jknd some sbifting of statements betweeh the tw fsctQrs . 

-^iqtTt-;BB="obWfVf^ a iT^fenE' aam^i eT^ ofHiginciMr ""r~ - \ 
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C are each described with 11 statefnents, five of which are cammon 
to both.. Nevertheless dua proco;.3 and fair practice are distinct 
ennuqh to be s^parstedN^n the evaiustion of accrediting agencies. 

Factors and F are ali related to difrerent aspects 

the evaluation procedures, ractcr D includes several general 
procedures for evaluation, -uch as initiating the first evaluation 
only at tha ehi^f executive offieer*s request, as well as state- 
/^ent?. dsseriwinq vi5i t ifi^ptt am procedurss* FBCtor E is related to 
the self-otudy * Foctor F. is a mixture o^' qualities that are not 
as cgherent n% those of tne other r actors; several of its state- 
met^t^ fhbout visiting team procedures see^ mors appropriate to 
racto^^. Factors Ci and I are sssociated respectively with 
orqnniiat .onal strength, .financial stftngth^ and preaccreditat^on 
procedures* . " - ^ 

THE CRITERIA AND FACTORS OF STATEMENTS 

Of. th^ 44 criteria^ 33 a/tt represented by 53 descriptive 

Slaten^ents orqanized in nine slightly overl&pping factors, ConSti- 
f ■ 

tulinq as they- do the major wa^ f* in which accrediting agencies 
^iffi s^-^own to differ in the DEAF repoits prepared for the Advisory 
Com.TiUee, the nine factors and their associated statements CQntain 
much df the informatibn on which recognitici diep^'^nds. 



Below, 1 n abbrev iated formj are the eleven criteria nut 
represented in the nine factors. They did not appear because 
statements associated with them did not vary much among agencies, 
^hey mav have been aiven' less attention by the DEAE evaluators 
than they merit; they may, even though important, be so readily 
satisfied that agencies were universally in compliance! or they may 
te so difficult to observe that little discriminatir^ information 
could be drawn from them, Several^-must notably a1i«-'are explicit 
enough that an agency may learn in preliminary discussions with 
DEAt stsf^ t^-t it does not at the moment qualify and may withdraw 
its petition before evaluation. Whatever the reasons ^ the follow- 
ing 11 criteria nay not, except in isolated instances, play an^ 
important role in the reconnition decisions. 



ali 



The anency is national or regional 
in scope. 



a1 i i 



The aqency clearly defines its scope 
of activities. . ^ 



a3i 



The agency clearly defines, each level 
of accreditation and has clearly written 
procedures for granting and denying 
accreditation at each level* 



b1 ii 



The agency's purposes and objectives 
are clearly defined* 



b2i iA 



The agency publishes the standards 
by which institutions or programs 
are evaluated. 



bZiiC 



The agency publishes the current 
status of institutions or programs 
and the <^ates of the next scheduled 
review. 



ERIC 
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b4 The agency fosters ethical practices 

concerning refunds and nondiicr unination 
in admissions and employment, 

b5 The agency evilijates iiis educational 

standards* ' , 

cl. The agency's policies^ procedure-s, 

and decisions are widely accfpted, 

c2 The agency regularly reviews its 

standards^ policieSi and procedures - 

c3 The agency has at least two years' 

experience as an accrediting agency. 

Of the above* Criteria bliij b2iiA* and b2iiC--all concerned 
with published inrormation about the aqency-rhad no more than two 
statement's associated with them that .appeared in as many as 10 
percent of the staff reports. In ef/ect, each of those three 
criteria was repf esented by little more than a statemfent asserting 
cor;priance.. From five to seven afrplifying or modifying statements 
were used occasipniily , but none was used frequently enougn to have 
a consistent effect on the decisions, Except for Criterion bZiiC, 
^here noncompl iance was found in about 10 percent of the cases, 
only two ;or three cases of nonGompliance with these three fairly 
uncomple^< criteria appeared in the more than 200 evaluations. 
While occasional failures to comply with these criteria may affect 
recoqnitian, they are not frequent enough to have an appreciable 
role-* ** 

Criteria ali^ ali-iV and a3i deal with the agencies' scope of 
operations and clarity of procedures. Each of these ? i 16 or 

\ ■ ^ 
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more statements associated Hith it, with from 5 to 12 appearing 
in at least 10 percent of the reports. The instances of mutual 
occurrence among those statements were not frequent or consistent 
enough, however, to indicate that they had much effect on the 
decision process. Like Criteria blii, b2iiA, and b2riC, these 
criteria were simply and directly observed^ with almost all 
aqencies complyinq. . . 

Criteria b4 and b5, concerned with ethical practices and the 
evaluation of standards , showed results similar to Criteria ali, 
alii, and a3d. Although moderate numbers o f relatpd descriptive 
statements appeared in the star f reports , they did not show any 
consistent pattern of mutual occurrence. These two criteria ^ whijh 
lie within the broad area of responsibility, are far more complex 
than Criteria ali, a1ii, and a3i, described abWve, which are 
associated wi tn f unctionil aspects. The complexity of te. and b5, 
and the absence of any strong ralitionship among the pertinent ^ 
statements, may sugqest that- the meaning and functions of these 
criteria need further conceptual clarification* 

Three of the four criteria associated with reliability are 
unrepresented in the nine factors. Criterion 'c3, requiring two 
years of accrediting experience, is simply defined, and one or 
two statements adequately describe how an agency does or does 
not satisfy it. It would be expected to appear in those factors 
affected by the length of an agency's aecreJiting experience-" 
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Its absence^ from the above factDrs is syrprising in view of its 
agsQcif tion with other igency attributes in the differentiation of 
agenciee described in Chapter Criteria c1 and c2 seem, like b4 
and b5, to be not well enough formulited to provide for consistent 
descriptions" of the ways in which agencies vary in their compliance 
or failure to cmplv* 

Several aspects of the ways in which statements were combined 
into the factors listed in Table 3 bear special consideration. 
First, there was some arbitrariness built into the statements 
themselves r whether two similar statements should. be retained as 
separate statements or merged into a single more general one was 
frequently deqided somewhat arbitrarily* The same problem, occurs 
in the recoqnition process, starting with the criteria themselves. 
Some criteria are highly specific, leaving little room for varia- 
tion or elaboration, as for example b3v, which simply calls for a 
meffiber of the visiting team to be present when the visiting team's 
report is evaluated. Other criteria are much broader and. allow for 
a variety of elaborative statements, for exmple c1, which is 
.concerned with an agency-s professional reputation and acceptance. 
An appreciable number of statements appeared in the staff reports 
under this criterion, but only a few occurred frequently enough to 
indicate that the information associated with it was consistent 
from agency to agency. Thus the criteria themselves range widely 
in their scope and specificity* The broader criteria, except 
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those which, like c1, elipit statements too varied to have a 
consistent effect, are most likely to produce the consistent 
variation required for such factors as appear in Table 3, 

In addition to the uncertainties inherint in the criteria, 
the staff members nriting the reports that .provide the Advisory 
Committee with a major part of its information necesssrily varied 
amonq themselves and over time in the degree of specificity with 
which they treated each criterion. Thif variition occurred even 
though staff members typically conferred in writing the report. 

Nevertheless, the criteria resulted from the extensive efforts 
Of a wide variety of peoplf^, and differences in the degree of 
specificity of^ different oriteria can be assumed to have some 
justification. Similarly, -'the DEAE staFf members were not wholly 

arbitrary in describing compliance with soma criteria in consider- 

" , ■ ■ . I,'- , 

able detail and giving dtberi .only a brief comment. Since the 
catalcg of statements reflected th^ content of the staff reports, 
where the reports were highly detailed comparatively large numbers 
Of statements were required to e^prtss the inforniation givenl 

The nine factors, despite errors, and some degree of arbitrari-' 
ness, represent combinations 'o f statements that tended to vary 
jointly,' They appeared or failed to appjear together in the staff 
'reports, and they showed some variability from one report to 
anc er. Yet the factors are not > eKcept in a very loQse sense, 
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generai attributes of accrediting agencies. For examples providing 
for, visiting ttame to meet with faculty , administrators i and 
students is not equivalent to giving the chief executive officer ah 



two conditions tend to occur jointly with each othe.r and with 
seyeral additional statements that have in common the protection 
of the interests of the evaluated institution or progrOT* This 
suggesti a general concern in the accreditation processi but not 
necessarily a persisting or completely coherent agency characters- 
is tic. ' \ ^ 

^Another caution may be needed in evaluating the criteria that 
are absent or poorfly represented. An important element in any 
particular decision may not appear in the nine fictors because 
it was described with only one or two statementsi and at the same 
time was not associated with other characteristics of' the agency. 
Some other important -element may not appear because. it was de- 
scribed at different times in different ways and so Ipst any 
appearance bf coherence. The' factors therefore may not include 
every issue of importance in the decisions. 



opportunity to respond to the visiting team's report* But those 



CRITERION FACTOPS 



Appropriate statements were sometimes omit^tW~^in aljme staff 



reports because they* were implied by other statements, wbile 
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several quite similar staLements were sometimes included in other 
staff reports. The unsystematic incluaion or exclusion of closely 
related statements intrDdui-es some error into the measures of 
relationships amona statementB, and into the nine factors derived 
from those relationships. This sciirce of error was reduced, and 
lh£f errects of .rarely occurring statements taken into accoant, 
b> factoring the criteria themselves . 

Each of the 559 statements was . classified as favorable, 
unfavorable, ormeutral in gccordarice, with the judgment each term' 
would imply about the effectivoness of an accreditir^ agency.' Two 
Educational Testing Service staff members made independent judg- 
mtnta, and a third was called on to rtsolya the few disigreements ,^ 
The favorabie or unravorable naturt of most statements was quite 
cliear; the few uneertsinticz; ^that arose . involved sMten.ents 
judged by one person to bf? neutral. Each statemfe^^ thafc spplied to 
a particulari-txatirion in a staff repoiL was scoced +1, 0, or -1, 
depandina on whether it . was favorable, neutral * or unfavorable . 
The scorer, for all the statements associated with a particuiar 
criterion wpre summed, a constant was added to make all the sums 
positive, and these were issigned to the ;agencies as criterion 
scores, S ix teen aq encies , f or which fewer than 20 statements 
appeared in the staff . reports ^ were ejccluded , and 44 criterion 
se^ores were assigned to each of 218 staff reports, Thesp 
criterion scores wer« then factored (Table 4), Since every 
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statiment is associated with at least one criterion, and every 
statement, however infrequently it may ^occur, can be assigned a 
value of favoratfele, neutral, or unfavorable, the criterion scores 
utilize the information in every statement. 

The first factor in Table 4 is a somewhat general one with 
which 23 of the 44 criteria showed correlations of *30 or higher. 
Agencies considered effertive in a number of areBs tended to be 
seen as effective in other areas as well. The first "10 criteria 
listed in the table are particularly likely to be strong character- 
. istics of th*? strongest agencies* The most general characteristic 
of the criteria strongly associated with the first factor is a 
concern for the interests of ell cooat ituencies , including the 
institutions and programs accredited. It is quite similar to the 
fir^t factor derived from the statementSj although that may be 
because i ^Dmewhst undifferentiated general characteristic under- 
lies the statements and the criteria. 

The second criterion factor has nine criteria correlated with 
.it.at values of .30 or higher, tlmost all concerned with the evalu- 
ation process. In this respect it is quite similar to Factor D of 
the stattment factors {^Table 3). 

The third criterion f actor involves two criteria concerned 
with reliability—experience and acceptance. The, requirement that 
fees be reasonable is weakly associated both with this and with the 
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TABLE 
Criterion Factors 

CrUerion Content ; ' Factor Loadings^ 

12 3 4 

ractor I ' . = 



0 t 1 



a2i 



Interests of aU constituehcies 71 01 n 0 ly 
are considered J ^ 

Personnel and procedures are ^ 71 35 ^^01 19 15 
fcrfective and timely. . / 

h'Hi Wlthdri^wsl of accrfditation 70 U?' 13 C5 ^ 10 

IS onl y for ciuse • ^ 

r4 Policy and riecision-mikinq 69 00 15 06 09 

' bodies reflect all constitu- - ^ 

enc ies . 



a2v 



Vlsitinq teams inciude nonstaff 66 17 01 07 ^04 
members, " - 



bJviiiB Hearings before an appeal body • 65 13 11 08 13 
?^re provided. 

d2 Procedures guard aqainst ^ 64 ^fj7 25 13^ 07 

conflicts of interest, ^ 

bZiiO fames' and affiliations of , 63 06 20 -03' 06 

-nembejs of policy and decision- . 
makinq bodies are publisbsd. 



b? 



Visiting team members, con- 60 21 -04 1U 05 

sultants. and member^ of policy 
and decision-making bodies are 
competent and selected without- 
discrimination by sex or race. 

Experifnentition and innovation 60 12 .^16 21 08 
are encouraged* 



bZiif " Description of ownership and 49 16 -02 18 07 
control is published. 
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table: 4" 



Criterion 



bio 



b8 



b3i 



i>3viiiC 
'a2i i 

b3v ii i A 
i/2iv 

.b2iiB 
factor II 



b5iii 

V 



Criterion Tactors 
(r;ont'd) 



Content 



Refirences to iccreditation 
staxus must be clear* 

Institutions or programs must 
meet published standards^ 
faipiy applied. 

An onqoing evaluation of 
accomplishment of goals' 
is required. 

Ethical practices are 
fostered. 

Initial evaluation is at the 
request of the chief executive 
officer , 

Written reports of appeal 
decisions a^e provided # 

The agency defines its own - 
fiscal needs and provides 
audited rinancial statements 

Accreditation may not be 
chgnged durirq an appeal^ 

Written procedures provide 
for -timely treatment of 
caniplaints. j 

Decision-making procedures 
are published. 



A written rtport qn strenqths 
and weaknesses is provided 
after an evaluation visit*/ 



Factor Loadings* 



1 

44 



t 
04 



3 
16 



39 



39 



39 33 



^08 



29 



29 2? 



25 



06 



5 
22 



39 t4 06 08 15 



02 -15 15 08 



09 15 



37 Oa 14-04 , 01 



31 28 • 27 11 19 



OU 04 09 



23 -03 02 -03 



20 07 13 01 06 



-04 66 -03 f}« -05 



TABLE 4 



Criterion Factors 
(eont *dj' 



Criterion , Content 



actor' Load ing 
2 3 4 



b3iv . The chief execu^iye ofricer his 29 ■11'' 13 

an opportunity to comment cn 
the visiting teams.* repcrt* 

Reevaluation occurs at reason- iu 51 11 -03 
cibie intervals. 



b3v A visiting team member is -U1 51 -18 . 11 

present at the evaluation 
of the report. 

- a?iiiB Written and consultative 24 50 11 05 

guidance art provided the - , . 
. ^^titution Of program and 
the visiting team. 

b3vii Reasons are given for adverse '35 42 03 09 
. '^decisions. 

^ faculty, adminiatfators J and ^ = 

students ? , 

a3il ' Apreaccreditation is related to 15 25 04 12 
accreditation. . 

aJiiiA The self-study assesses 09 Ibf -02 13 

st.renqtha and weaknesses ano 
' involves all constituencies. 

Factor III 

c3 The agency has at least two 12 MO 75 00 

-years' .e><perience* 

c1 Policies and procedures are 06 17 .58 10 

widely accepted, 

a2iii ' Fees are reasonable. - 22 45 20 




^ . 54 

TABLE k ' 

Criterion Factora 
^ ■ -.' (cont'd) 

' Criterion ' Content . Factor Loadings^ 

■ 1 ' 2 3 4 5 

i S'd f unc t ion * f* ^ r s w ' L - ^ ^ ^ ' 2 ? 2^ ^ ^ D 

the exeraise of indtpendent 
judgment *- 

F actor IV 

^ e2 Standards, and procedures ar^ . 13 14 1 1 68 21 

regularly reviet^ed. 

b21ii . Advanct netice is givtn of 34 04 04 62 -05 

rtvisions of stindards* 

b5 Standards are tvaluattd for 19 .11 11 40 24; 

validity Bnd reliability. *■ / . 

\ ht i xt Current /iceredilit ion . 05 00 02 31 -20 

^ statuses and dates nf next 
review are pub^lished, 

bZiiA Standards are publiihed* 01 .01 -10 21 07 

F actor V . . ^ ^ 

blii Pufpoies and objectives m 09 . JO 1CJ 64 

^ 'are clearly ^efinedi. ' 

alii - GeDqraphic and eaucatlonal 26 08 05 24 52 

scope arei^clearly defined. 

Not Associated with a Factor 

ali The aqeney is national or 08 -06 -03 10 00 

s regional in scope* 

a3i Oefirtitions and procedures 01 06 -04 05 02 

are clearly statid« 

b2i N . The public is reprtsented on -02 05 -13 04 -04 
policy and decision-making 

bodies* ^ - t 

Oecimai points are OTitted. 
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. first factor* The two criteria mpit strongly associsttd with the 
third factor did not appear at mH in. the stattfntnt factors . That 
the most e.HperWnced agencies should also be the ones most widely 
accepted is not surprising, TheBe qualities may not have appeared 
in t.hs fac'torE nf :A ^mpntn bpr^wsp th^ \f^v inn nn ^vp^^rifnrA 
hod eomparat ively few itatements associated i^ith it, and although 
th^ criterion^ on acceptance produceo a sizea:jie number of statt- 
menti :!17:, only four appeared in more than 10 percent of the 
reports. Those four were not itronqiy enough related to one 
Bnov^^t to affect the- fsctor structure of the ststemenls^ y.et 
they and tht 1 3 inf requent ly used atatemerits apparently produced 
scores ^or the criterion on acceptance that were rellsble enouqh to- 
show 0 relet icnsHip with e^Deriencit, 

The fourth cfiterion factor is associated with the malnttn^nce 
and evaluatisn of standards. Of five criteria directly related^to^ 
itandafds^ four had their highest laadinga on this factor* The 
, fifth, b8, .whieh is oonccrnod with the fairness with wt^ich stan- 
dards are applied, was weakly associated ^^^ith the first factor but 
not ^t ^11 with the fourU^. 

The absence of a fact:5r in tha statements associated rf^ith 
standards is surprising* One such factor had appeared io a 
j^r^n^inpry factor analysis' of the early set or. 52 staff repocts 
from which the statGrnentB were drawn, ^et the later analyses 



or more than '200 reporfcs failed to show that factor, even though 
the five etittrla eonceffiad with standards iholuded a total of 50 
diffeftnt statamentSv Of thtse SO, 20 were eniered into the factor 
analyses Rathtr than forming a, coherent faptorf however, these 
statements tended to scatter ampng the otheA factors^ primarily 
the one related to fairness. Statements about the fairness with 
which, standards are applied r the consistency with which decisions 
reflect the aQcncy'a standards^ the clarity of the standards and 
its role In ensuring fair treatment i and the ins^olvement of all 
eonstituencies in the revision of stendards were all associated in 
Factdr C with otheT statements reflecting fairnessand integrity* 

' Functions related to standards are reflected in five of the 
44 criteria, four of which show their Interralatedness in the- 
four Lh cri teTion factor. That the statements Concerned with 
standards did not themselves form a distinct factor , but instead 
spciad Over several , other factors, raises quastions about the 
pertinence of the observations that lead to the judgments about 
compl iance^i th the criteria on standards. One possibility is 
that thfr criteria may not have been defined clearly enough^ and 



ano^tfter is^ that .the inherent difficulties in maRinp^ the necessary 
obsef^ations about ^standards are so great that fthey preclude the 
devflopment of statements that will show the expected relation- 
ships,' This issue is developed further in Chapter VII. . 
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The fifth criterion factor included pnly two criteri|, both 
concerned with, .the agencies* clarity of purpQee and icope. Neither 
is rej^^iented in the statement factori, probably because of their 
specificity. Only a few atatementa about the compliance or non- 
coiTipliance of, agencies with these two criteria were made with any 
appreciable frequency. 



IMPLICATIONS OF THE FACTORS OF STATEMENTS'^ AND CRITERIA 



The two -^.ets of factori--the factors baaed on the atate- 
ments directly and those baaed on criterion scores derived from 
the^ st atementS"-lead to aeveral sets ol" judgments about the 
criteria^ Jhey defihe the content of the criteria in terms of 
the observations on ^hich judgments about/ in agency ability to 
determine ''educational quality a,re based/ Fair application of 



standards, for example, which is a parp of Criterion b8, also 
incorporates elements from other criteria/ It impHeSj in addition 
to decisiona that arfe consistent with standards and fair procedures 
(Criterion b8 ) , clear definitlonr. of/standards f Criterion bZiiA) 
and protection againat conflicts o^> interest (Criteripn dl) or 
other forces that might Interfei/a with independent judgments 
(Criterion d2)* The overlapping of aeveral criteria is not 
undesirable, but rather demonitfatis relationships among iasues 
that should be conaidered in Judgments about an agency's compliance 
with any particular criterion, • ' 



■58 

The two s#t^ of faotors point up pottntial gape or weak*^ 
n&BT^WB in the* ^plioation of the oriteria, Critefla that wtrt not 
assoeiated with at leaat one let of factors ware probably not 
contributing to tr-^ necessary discriminations among aggnciea* Only 
three orlteria ware missing from both sets of factors^ralii^Sit 
and b^i^ — which were conc^rnfd respectively, with the agency's 
geographic seope, tne clari,ty-Qr its acorediting procedures^ 
and represtntation. of the public on its policy and decision*making 
bodies* These three were mlssirtg from the factor!^ because almost 
all agencies complied with them.Mthout question; Although they 
were considered in the evaluations of the agenblasj they did not 
provide enough viriibility to CQntrlbute except occasionally to the , 
dlscTlminltioris among agencies* . " " . 

CriteriT c1| c2| and c3 ware missing from the factors based 
d\r;ectly on is.^ a tements , biit riot from the criterion factors.* 
Criterion e3^ requiring at least two years V experience, was met 
by V'irtuaJ \y every agency petitioning for recognition and t jquires 
no el aliorttion* Criteria c1 and c2, however*^ are less cltear. 
Criterion cl calls for wide acceptance of the i^'enny*s policies 
and proeedureSj and Critet^iof: c2 requlrefi regular review of the 
ag#^ncy*s standards to m%Mta that they are const ruc^tivei important y - 
and pertinent to educational qual ity# • Earn had an appreciablr : 
number of statements ss^^ociated with it, but the statements were 



not aasoeiated with each other sfcrongly enough to contribute to 
Vs t at ement -based fac tuff. This suggests some uhcertalnty about how 
to 6bserv6 or evaluate Bither wide acceptan or tht strength of 
an aqtncy^B program to keep its itandardi current and relevant to 
educational effectiveness. 

.• Criteria b4, b5, b6, and b7 appear,, with the exception of ba, 
in both sets of factorsi all cbntribute to the discrimination .among 
iqencies. Critefion bft, CQncerned ' with tht fostering of ethical 
praeticts, is weakly associated with the'genefal factor among 
. the criteria, the factor that reflects the agenoy's; general effec- 
tiveness, rttspo'nsiveneas , and atten't ipn to. due pfoctss. Like 
Criteria c1 and c2, the ethical practioe'eritw^ may require 

greater, specification of the kinds of observations that will 
indicate compliance, the other criteria mentioned above are 
related to the agency's evaluation of its standards for validity 
and reliabiiily {b5)i attentian to the inptitut ion's or program's 
©valuation of its educational product (b6), and ancoug^ement of 
expefimentation and innovation (b?). All have statemanti that ar^^ 
scattef'^d among the faetors, suggesting either^ that those criteria 
aKauld be clarified and appropriate kinds of observation identi* 
fied, OT that they should be incorporated 4ntb the other criteria 
With which they are related* Criterion bS, for example, con- 
cerrang the asseisment of the validity and reliability of the 
agency's standards^ might Nell be made a part of Criterion c2, 
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which involvts the regulir revitw of the agtncy's standirds 

/ ^ ' -. - . 

and pyocedurea, . The evsluat i^n of standards wpuld still need 

elsrlf ieatlon, but atsociating that proeeas with a reylew of 

tha affectivtness of thi evaluative proctis as a whole might aid in 

that olarifieation. ^ 1.; 

The two sets of factors also mjggest^^^s^^^ quiitlonss Is the 

scope or covex^ig^ of the criteria lufficlently broad? And do the 

-crLter ia Incorporate Mlt the important oonsiderationi that should 

tnter into dett rRiinat iohs of agtntfy t f fectivf ness in judging 

tdueational quality? Both sets of factors help define the existing 

eriteria, thus aiding in the ide'ntification of neglected areas of 

importance. For 0xmplWf criteria associated with the responsive- 

ness of. the agency to its. various constituenciei do not include 

issues of consumer protection. A^hoygh there is' no consenpus 

about whether this matter is a proper concern of aCDrediting 

iggnci'es, its absence from the factor assoqiated with respoihsilv.e- 

ness, where ii would be expected to appear If included. in the 

criteria, is clear. ^ 

" . _ . - ■ ' ^ 

Another judgment tha.t^can be made on the basis of the factor 

analyses involves the organisation or structure of the criteria. 

identification of the system of rerationshipa among the criteria, 

which the factor analyses provide^ can aid in clarifying the 

meaninq of tha criteria generally* 
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Much of the systematic vfriability in aganeies, the faasba far 
diacrlminationa among them, is described in the ractor 
resul ts . The variabil ity is asspciited ^with the opennes 
agency's, structure and ■prQcedures to public view, its responiive- 

■' ness to the interests of the. agency's various constituencies, ite 
exercise of due process in accreditation decisions, and its fair 
application of eyaluative standards arid procedures. These ^our 

"phrases account for six of , the nine factors into which the deserip- 



, . . . ■ - , . ' ■ •■ ■•■]■ 

tive statements can be organized, and for the, first two of the 

criterion, factors. Some additional, variabirity, repres'ehted in 

the last three \statement factors, is asspciated with orginj.aatlonai 

strenf;tn, financial responsibilitf , and preaccreditation proce- 

1 . ■ ' ■ , " ■ ■ ■ . ■. 1 . ' ■ ■ 

dures , phe three additional criterion factors are "concerned 
with experience and acceptance, the review and, revision of stan- 
aards, and definition of purposes and objectives. 

■ - ■ - . t = ■ - . ■ ^ ■ . . t . •■ ^ i 

The description of the strLctup and content of the criteria 
as they, art^ appJ led is important for two reasons. First, -dt 
reveals the degree to, which the dlacriminations among agencies 
reflect the fuil range of the criteria. In this respect the 
reptognitioh process per forms quite well- All the criteria ej^cept" 
tnree contribute to "the di at inct ions, among the agencies as they 
are described to the GommissionBr^a Advlsbry Committee. The three 
except ions are alj: explicit criteria,, simply met and readily 



obitrved, whieh prevented them from contributing to the vari* 
ability among the agencies. Second, it points but areas wi thin the 
criteria that lack conceptual clarity. Here somt problems with the 
-tftiteria were observed. ° ' V 

Two rtiatad critical, questions will be dealt with In aubsa* 
quent sections of this report One is whethtr the distinctions 
among 'the agencies described in the staff reports to the Ad^isury 
Committee were appropriately and ponsistehtly reflected in the 
recognit ion deciaians . The otKer question is whether thejpe. was 
aqreement among k>io wledgesble persons, that the distinctions 
among agencies that determine, the granting or denial of recognition 
are approprista Indicators of an agenay-s ability to evalu^vte 
educational quality, ^ . \ 

A BCSTRUCTURING OF THE CRITERIA ^ ■ - ' 

. . A suggestivf grouping uf the criteria J prtff^^^^ iwlow, was 
dfetived from the factor &halyses ^ Judgments d»' U^e imp:u:;tahce of 
many of the statements' (Chapter Vl), a geometrir repr^s^fn^t^itldn of 
the distinctions that appeared among the igencle^ Cl^h^^ Isr V), and 
f rom stAjecti^^e ^ense of the cont the, ^^Iteria* Thai 

ar^anqement places the, existing criteria into thre^ broad, areas— 
organizatiof? and sconi?* accrediting procedures, iind teeponsibiritv* 
fNese three areas are cancerned^ reHp«ct ivt»ly '^with an agt'icy's 
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operatiorial ,8truetufe, purpose, and 'context} tht procedur^R it usea 
M evaluating and acoredlting prpgrams or inatitutlons? Mif i,he way 
it txtrciies its reaponaibillty to its varioga cominhmi ties of 
iriterest, iheluding the general publ icy The areas difrtji^ in adopt/ 
moving praqreesiitly Fr^m internal characteriitics of *;n®^®gency to 
concerns for its immediate const ituaoeyi" the progrw^ ar. Institu^ 
tion^ with v^^^ich it deils*-tp concerna ^r its brmcter conatitu- 
encies • Cach >r6ad area -iE . elaborated into th^m mo dat«lied 
sets of Qritaria^ The criteria are paraphrased for brevity, with 



their presant dgsignationi given in paranthesas. 

;■; i ^ . ^ " / ^ ' : : - \-/ 

I* OrgifnlMtion and scope 

A* Scope and clarity of pur pose 

' ^ 1* Geographic scope is'nitional or raq^on^) 
.Jell). ^ / 

2, Functional scope and purpoaas are f;»L^arly 
defined (r'liij b1ii)* ^ : 

E- Orcjani^ational strength 

' ^ ' >■ - I ' ■ ' ■ . 

^ 1. Staff is competent and luau e^rvough for the 

damands placed on it (a2i). 1 

2^ Viiiiting teams^ CQnsyltantSi and board 
' ft^embers are qualified (a2iy),^ ^ 

1« Policies and j^roeadurH .T»re widely accepted 

\. . . . ' ' - "'^ ' . ' ' ' ^- '" ' 

^* Jh.^ .egtncy has j^t least two years' e^perieTOe^ 
Flnar cial' strangth \ / 

* '' ' ... ■ V. ■ ■ * 

.1^ f inancial ^resources are. adequate ( 82ii) * 
. Fees are reasonabla (aiiii)* 



Accrediting proqaauffrs ' - ^ > 

A V AecTa d I t a ti on - 

^ 1* Accrgdlting procedures are clesrly defined 
, {0L)y : - - . : ^ ^' ' ; .." 

-li Preaeer aditatiQri is releted to iccraditation 
(aJii). ' y 

3. Written and oqneultativt guidance 4a pcjo* 
. vld&d to programs or institutiws (a3iiiB)* 

B. Evaluation . ^ 

a* A aelf-stydy is required Ca3iii),. , ' 

b.^ Thf sel f*study evaluatea the strengths 
and weikneasaa of the program or 
^4nstitutioni (aJiliA), . ^ 



cv The agency gathers information showing 
/ a Dontihulnq program of eval^atiort of 



outputs (b6). 



2* Site visit 



a* 



An on^sitfe visit ia requir&d (aJiil ) * 



Visiting team membere art, knowledgaiblt 
and qualified (i2iv^). 



Visiting teams are qiy^n written arttf 
consultative guidance \a3iiiB). 



, ^ d. 



yisitijriq[ teams meet with facul ty^; stu* 
dtnts and admiriistr'&tive staff ib^ii)^ 



Standards 



1 . A prdgcam to evaluate the agency's standards 



is maintained (b5| cE). 



2* ^ Resctl0he to proposed revisions of standafdi^ 
^ are solicit^ (b2iii)*" , ' ; 




3. Only quQiified programs or insWtotions-are 
aberedlted (bi). - ^ 

Raiponsibility . \ ] 

A.\ Responsiveness , , 

1. The Intereit^ of ail Sonstituenciis and the 
qanaral public are considered (bli). / 

a. All direotly afFepted cons tituenqlas 
are represented oh the qsvMning 

board (eft) * 

b. Repreatntatiyes of the public are on the 
\ governirig board (b2ljt 

2* Standards, procedures , and orgspiiat iorial 
' infpmation- are published (b2il)* 

3. The agency fosf^ers ethical practices Cb4), 

4. Experimentation and innovation are en- 
, couraged (b7), " \ 

5. References to aocreditatlon must be clear 
CblO). ; 

t> Due process 

■ ^ !• Written profeedures^^rovide for equitable 
V review of complaints against prog^ or 
\ institutions (bZivK - . ' ^ 

2* Accreditation is i^ithdrawn o cause 
and after due process CbSvl), > 

3* Specific jreasqris for unfavprsble decisions , 
and notification of the right too^pefrl are 
given tbSvilK ^ ^ i 

Appeal procedures are provided (bjvlil)/ 

. FairnesB, impar tialit^r ' 

" 1. Standards andVprocedyres 

decisions are impartiai and Iridepe^^^^ of 



. a* Operating procadurea protect against 

, conflicts of interest (d1, d2). 

b. Visiting teams inelude at laast one 
parson not connactad with' the agency 
, ; ( a2v). . 

All the eurrtnt criteria ire included in the above list, 
in several cises eieinents from two d'iffersnt erlteria have been 
combihed, such as ba and d1 , botK- concerned with impartiality. In 
othtr cases a Einglt existing criterion^has been split, with parts 
of it assigned to different criteria. Criterion a3iii, for ex- 
' aniple, has been separated into those components associated with the > 
self-study and the sit-e visit. 

' : If tha suggested structura is to be useful, the components at 
ateh hierarpfriical lavsl should be roughly aquivalent in scope: or 
Impprtincaj with the higher levels, encompassing all tha issaes 
with which rac'^ogniliDn^ should ba concerned. T^a lower-level 
ct5mponents should specify the scope and content of the, broad 
Issues. Periodic modifications in the crita^ be expected 

primarily at the /lower lavals. Graatar alaboraUon would Incraase 
both the raliWbility and validity of the judgrnents about the 
higher-order criteria, particularly those that are difficult to 



• . V-. DIFFERENTIATION OF AGENCIES 

, Recognition decisiona art intended to difrerantiate smorig 
acertditing aganciei in terma of their ef fectivantss as fvaluatorg 
of t4ucahlonal qullity;* The validity of thola depisiona lies in 

^ thf tmpirical juitifiDatibn for inferring that different degreta of 
strength in educational evaluation are aaaociattd with different 

. decisiona. To asseaa the validity of the^ recognition prociaSt 

there fore t one step was to identify the diatinctidns imong^ sccred- 

iting and itate approval igtncies deacr4bad in the DEAE staff 

- reports--the distinctions o,n which recoqnlti^on decisions ari 
* * 

largely based* 

In one;sensej the published or iter ia define the . pertinent 
differencea among agencies^ and the more detailed atatements drawn 
from the reports provide further elaboration. The clarity of the 
diatinctions ia reducedi however, by the redundancy in the uriteria 
and the atatementi. For example, Criterion bS requires fair 
application oP agency atandards and impartial accrediting deci- 
sions , Criterion d1 requires agencies to be autonomous enot^h to 
produce' "independent Ju^ment(s) of the Quality of an educetional 
program or ir.^titution." Diacriminatlons among agencies can depend 
on whether £4-.eae criteria describe dlffererit characttEistics , or 

^ 67 . ■ 



¥»hetK#r impartial tfeciaions and fiir application of iltandards 
necessarily/ imply Ihalt judgmftfrta ar^ r^chfed independtntly oT 
asternal pressurfes* * > ; ""^X^^ 

PROCEDURES FOR' OESCR IB INC AKNCY DIFrCRENCSS '• - 

Because tha honredundsht distinetwns consistrhtly pQund 
in the Stafr reporto reflect the Ktual appliciticin of the critefia 
and prov Idtv^much of .th# q round for the rtcc^nition deeisicns> if^ 
.pracedure was reouired to idei^ify ar^ describe them/. Two compi^- 
ment ary precednjres ware fo^owed^ one more* iensltl^re to the large 
distinctlonst and tht otSver mjore'^sensitivs to the smalierf more- 



minor distinctions* Botly^procidures start ftm measurements of the' 
overall deg.^ee of similarity or difference mong the .objects being 
e\iimined* Only the first prpoedure, multidimeneional leslitigs, is 
described. The second j cluster analysis * because it focusys on the 
minor distinctions amon^ agenoltSf formed small cT/ysters which 
showed little geneta^lity from one analysio to ahbther* Instead, 
they seemed to reflect whatever qualities happened to be CDmmon to 

. ; ^ ■ ' , ' ' ' , ■ ' ^ ■ 

a fe^ aoencies in the particular group beli^ anflyted* 
smail clusl;*rs were fofflied into lerger oneSf they agreed ¥^1 1 with 
tKe spacmQ l agfencies in the first typt of analysls--multldime^- 
slo^al seal inq CMDS)?. Therefor^ #irtDe MOS is more descriptive 
than cluster analvsis of the large di f^ferencesr among ^encieSr 
slnc# *th^ small differencea em^asiied by fluster analysis were &t 



stable ^From one snaiysls to anottiar^ and aince^ the 4arg£* elusttra 
supported but did not add £e the inf ormaticyfl in the multidiflien- 
si onsjl seating analygiSp only the MDS results ar#^repprted. 



rhe MDS, Procedure 



On^e the ^^verall difference OTong a set\pf objMts hive been, 
eKpresstd quantitatively, myl'tidimerisl^^ scai^llng will identify a 
smal I number of dlmenslQjmJIfset aCcpmt^ fo^ rnuch of \the v^ri^- 
lion among the objects, In,a iucldj general dedc rip t^^^ multl- 
d imfrns ionil seal Inq ^ Subkov iak ( 19.751 derrrified how a eet ^f 
quesses peopJe made about the compa^^ti^/e distances between 

' ' . . ' ■ - A ' ^ - ' - 

American cities could be analywdrto ptrotji^e a moderately accurate^ 

■ ■ V V ^ ..^--^ " ^ ! -v.. , \ 'Y - . ^ 

map of the Uiiited Statei. HBnf judged distanoea oeiHi^en pairs of 
cities were reduced to two dimensions on ^hith fthe citiii were 
placed In iQcat ions that welTt maCehed their actcjai gfographic 
I'oeat tons* Similarly*;; measure of the overall di frtereneei eifRong - 

• ^ ^ ^ ' ^ L \ 1 

the agencies^ derived froni. the WA£ staff reports^ led\ to |a mapping 
of the agenclfs onto a few dimensions that actojont fdr\much of the 
systemat ie variation In agency charaeteristics/ Their origin in 
the staff repcrts made thuae dimensions reflect the majbr distinc- 
tions amc^ agencies thai commonly enter 'the recognition yepisions* 



While the spatial representation of thte agencies re^^ealed^e 
mopt commonly applied bases for the tecognit ion" decisions? t^^ 
information it arganiied from the staff reports wasLnelthef 
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. nor entirc?!v accurate, Aqency representatives provi'ed 

iUjpp 1 £»fTi?*nt ar y information at Advisory Committee meetinqs that had 
not appc^ared in the staff reports. Such belated informations of 
cQuvm* could, no^ appe^r^n the htOS results* That its omission has 
not distorlg^d the-^DS representation of the actual decisions is 
!:.iKu;rGt??a D tne fact tnat tne Advisory Committee has recommended 
;ierual nf reccqnitiDn three times as often as has the DEAE staff, 
I f t^e c>wpp I e^^pnt ar y information from the agencies had had an 
- f pr ^ ?t^7 ] pffecf on rhp decisions, the Committee's recommenda- 
ti'.nm wc;^ld have been more favorable than the staff's, 

i ■ 

Another kind of information lost in thp MDS analyses consists 
of the unique or unusual ways that an agency might fail to meet 
one or ^wr^ the criteria. ' The differences among agencies that 
iifn^idf^r^ fhp sfartinq point for the MD3 finRlyses depended on 
the relative number of commori€y used descriptive statements that 
Jint inqyished one of a pair of aqencicj from the other. Statements 
that appeared rarely in the staff reports did not have any great 
efrect on the measured difftrences. For example, the following 
f.tatements appeared frofp one to four times in the 234 staff reports 
anal y zed: 

So systeniatic review of instruction, resources, 
management, or student services is done which 
results in a written report*. 

. Standards by which the quaj.-tt'y^ of programs is 
judged pre ambiguous fij uhciear. 
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The agency is gehei=;jlly judged to be unreliable; 
its actions are :nn55id^:^rf questianable , 

The accreditation conf^if^-d by the Bqency has been 
distorted by politicui] exp^Jdiency* 

Any of these agency characteristics might have bran Influential in 
s particular decision to deny recQqnition* Yet because of their 
infreGuent appearance, they did not have any aDpr^ciable affect 
on the MDS results ^ which reflect commonly observed difrerences 
among oqencies* 

The four statements cited above appeared in ten different 
'^staff reports, with no more than one of tjn&m appearing in any 
L ^qle report. Six of the tten decisions were to deny recognition. 
The other , four were for recognition for one, two, three, and four 

years. . , 

Important information about particular decJ^tons^ is clearly 
lost in anaf^yses based on cc^f^ ^ dlf-f^fenc^s -a^^ung agcncl'^tt, tincr 
/rarely occurr ing qual.it1es may be ^ influential in the decision but 
not i^J^Ktr an^ lyses * This lost information is not critical, 
however, because the study is concerned with the natMre of the 
recognition prqeess as a whole. Of , interest are the quHliUes that 
conslstpntly discriminate amonq eqencie*! rather thin thcfte that 
only rarely affect a decision* In every staff report it^ wnich.one 
of the. above Btatempnts appeared, a nurnber bf o/ther statement i 
appeared that not only providtd a more qmplele^ coht^renl view of 



r e nq^ncy, but «ere rp;.^atf^d rruqu^ntly (*r=ouqH m reparti on othyr 
moncim to have m'sfftict on the MDS results. 

Jhe differencm bet wen pvirs o?^ aqencics* from which the HD5 
anaUses Htnrtt^d, were the inverse of measure of similarity, 
'hat -rr p;n^urp vf3S hA-;*>d nn r hf^ nrnnnrtinf^ nf H^/^ tnf^I n?i?^h^^r f 
□ tntements^rndK ribout anv pair oT agencies that wifre co^mnun to 
bo'n*. rnr e<anpiff» if 30 statements wore used to describe both 
rin^ 'Cie^^ ir a pair» and iT each aqency hnd an additional 30 state- 
r.r^nts tfu^a dPGcribffd it buv not the other, half of. each agency's 
f*?-0cr;pt ive chapact^nst ics would be shared with the other ^eney* 

the proporUonn of the m;mcx^B' shtrtd qualitAes Ancreaie, the 
a^^wclf?^ similarity imtemQn and thrir difrerence decre3se:s, the 
r^oAt^ur*^ uned to qui^nt i Ty the Similarity ^^tiung nqenc ias was k nppg 
>f leisfv* it takes into accomt tne tatml nunber or possible 

stateipentn that can be madp nbout m ageney--:n this study 559--and 
arfiu^ts the r^Mf^uri? or simiisfity for tht: number of stat j^nts that 
«roul(l occur m cotrmon by chance. Ihus instead of a eimilarity of 
*'^0 t>etof>-n thf? aqenries in the. above Q^mple^ Inoicui^ g that half 
D/f f'fich nqj^rvcy r,^scr ijcd qua! ities were shared >*ith thu olhct ^ 
ii^p ^imi2iirit> c^lcul^ted kappn was .4^* That reduction ac- 
rnunlrj for the fmt 'hat SQmo small numbeT of common statomehts 
*#oyid gppoar in t^o fsnmples or BO drawn randocply from a total of ^ 
5!^?/ si^rilarity inlcrprcted as closeness" and dissimilarity 
or difference aa ^eparftion or distsnctf, agencie?* will be clort© 



lo^oth^r "Ker: ihv^ hn^f^ . e«?n d^scriU€*d it., similar phrasfes rhc 
DlAl ^iB^f ff^ports* ^^qer^cies dt?scribRd in tfjrtns that have lUtlo 
m co;r^:r<w.^i can be considered to .of? ^idf/ly separated or distanL. 
"Tter tMf? dif^^ance toetp^een ev^^ry pouf:iblD pair □/ sqencies i^cjs 
tieterfT^ ir Gd , tnt aqdncies ^t?re placed, throunh^^ il t id irnens innal 
scaling, in a space of several di/nensions. The number of c. -n* 
s.jnfl required to accommodate all the agencies, ^ith relatively 
little ^U^itorti-n of the iininnc&^ between tncm^ indicated the 
r^u^Ui'V 0'' indf?p^ndent fsetig of .d ist incLions between aqencxos* Tne 
char s^'tc^r itit irrn of thp ngenrie^ Ir;/ .f^d ^it f^vtreme positio^^a on 
those dinentilon^ then indicated the nature of the distinctions 

4 , 

that had tpen /Tiace. ^. ^ ' 

jfie wijy n >*tUcJ^ tne d. -taress betwr^cn ngRncies determined the 
nu'f^ber of di^ensionB needed tn oc^ommodsti thefti is illustrated in 
f iqajre 1. IN^ fiistfincns between pn?nt;. A lujd B and points 0 and C 
a: rf>spect 1 v€^l y 3 and 4 units in both, parts of the figure. In 
fiqure r a; thP di%t^inco between points A and C is 7 uhitsi 
in finur^ rb' it is If oriy thp Ud sets of distances are 

krmwn, t ? vn r 'pt f-'^f^nt ^ iQnn nf r iqure 1 c?in bt^ picttod,;Gne 
rtfqyirinq Dni> one dimeoi^Qri and one reQUirinq two. 
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^ ^ plots of thrne points 

In problems irwolving estimates of distance or cJosentsSf 
as is the case in mensuring similarity between t^io ageficits in 
tetms o r the proportion of their com^monjy shared dtscriptJvt 
St atementB I ror makes detecfnination wf the number of dimensions 
imptaclie. Judgment must thtrefore be exercised to deter ,ne how 
mmy dimensions represent a usaful erdtring of the, objects and +>ow 
many are simply the result of error in thr estilnates of distsiee* 

s? ■; 



in the present study, at least tw^ imensions r e inv Lab i y 
required to represent the differences arriong aqsncies deacribed in 
the staff reports. For some q roups , Ihrrf? dimensicna gsve s more 
reasonable picture.. 

as the number of aqencies analv^ed ;s inereasec^ Th#; ^^r of 
nqenci^B included in any sirqle analysis His th^jrefgri? lim.r 1 to 
^bout 50. Tne a-iencies were grouped for the ?0S analyses occorc, 
to the aet f rriteria b^inq oppii- thgjm, The/reqio?>\ 

ciqencies tnai arcredit coHeges and univer^^t^^s as total instH';' 
tions and thi? special iz&d aganeies that ??^i;:. .:tl occupationai ; 
proressional pf^rams, or in some cases spe^: instilut lons^ 

wi^re txamin^d in-r^iatiDn to. two s^ta of cr ^ : i^? -tKe criteria 
puDiisntd in 1^69 and the rtvisJo^. A few sfj^nc;ies= in 1?d8 

anl early ^969 had the pro*1969 crUerifi appl led, but Ihey mvw Jboo 
few to justify separate t: tmont. The stale feo^rde of appcDval 
for voeational gducstion a? state boards o f nu= sinq were 

recoqnl^^d under two slightly 'ferent sets of criteria. Thus 
-fowr nalurel qroupinqs for analysia '-^-i 'liEleri- -wj'jneips ev8lual^*sJ 
upd^r the 1969 criter.^. the i9?4 eriieri&v t^^s nursif^ criteria^ 
and e vocaliaRsi c^inrria- Wh^n all ag^fcies were analysed 
tog ether /the result a two-dimensionaV apif?e With the compsfa* 
lively Urye nurmber af rtrqionai and special i^e^ Mrnciee near v 
center of the s ice, the sffire vocational boar-Is clustered at the 



enci o^ i^ drnencAion, and ttm ptate nursing boards clustered at 
fnd the other dimensiDnp . the natu? u of the criteria applitcf 
pfoduced such great dislinctibns in tm descr iptionsi^ f diff^tant 
types of aqenciQB thai differences ammq the agencies within thqse 
thr^'e qroups wtrt lost* 

?hi^ aqejieies that rnado up iHe largeot single graup were 
tn^^ feqional and specialized agencies evalu^it^d according to the 
197A criti?ri3. Btrrause the agencrias ^yaluated in 1974 couid have 
had tilh^f the 1?69 or 1974 criliiifia applied to them, 8£ reqional 
and fipeci3li/^d agencies eval uated f rom 1975 to 1973 were anaiy^Lrd 
in rAnf|i>^Jy formed qraups of 43 aqwcits each. 

Tigurf 2 shsw^ a group of 43 regional snd ^peciRl^^ed 
iigi^nei es* evaluated b^tv^en 197^ and 1978, .^t^ieh have been plot : vd 
two of three dimenBiDno dov^vud from nn HUS analysis of thctr 
similafitiei and differences as described i^i th© D£ staff 
repofts, ^ Cacfv number or li^tter in Ihe ^*gure indicst(?g the 
recognition decision for . a partieylar a^encyt mc its Mprtipn in 
the t##o-dsmensional opace ind icatea the sgencv*^ chariclariatics 
with respect to IhouQ tm dimensions in relation tn the othor 42 
aqenci^s* Tl"^ nwnbers 1 to 4 indieste recognitian ^or that nimh^/ 
of years. The letter p indicates denial of recogrtiticn, SC a 
raquesL fdr th© ^aqgncy to show eiuse why itf recoonltion should 
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not be revoked, and WD (in figure 3} an aqt-ncy that .wittidrew its 
petition Far feccqnition. . . 

With a ^^ceptions, the aq^ncifrs ^ecognired for fsfjr years 

recoqnized fcf oniy ono year are at the top or scattered around 
tho periphery of the figure* Fhe agencies in the cej^ter: of the 
fiqure are described in the staff reports by a Jarqe nii^ber of 
favorable s*:atements incJ icat ing compl lance wjth virtually all of 
Ih^ criteria* 

The 3tBt^^eni*i most commDnly applied to the agencies at the 
top of the vertical dimemion questiof^ed the impartialny of tneir 
decisiona, their respgnpiven^ss to the interests of all their 
eonot rtuenc ies nnd the general public, and their interest in 
maint ainincj acceptable standards. Of* the six aqenciee cloBest to 
Wp' top of tne figu^f't fsut ne re ^denied r^i^ognitiOT and im Here 
r!^-rr*qn:^ed for one year. Ench of the follo^^nq sUtements appeared 
in !he staff rt^ports of at least four of those, six, 

^onc? doi^t gxists as to ^•^ther th agenrry ensures 
impart ii?^ and objective eviluations and dreislons. 

One Of more const i tuent-ie^ ^re om ; ced or tjnder- 
represent td on the agency's poli^.; or decision- i^ 
mnkanq body. 

Th^ agency does not provide for or cufrently have 
public represantativtt on it;s poUcy or decision- 

' ^ making body. ' * 

, *■ 

The aqencv does not oiAlish the eurfSM acefedit«- 
tion otatus of l^jtitullona of, prolans. " / 



ihif iiq^ncy does not puhiish diiti^tj for t?># m%l 
revi(^« of adcridUed institutions ar proRfaf 3,^ 

Appeal proc^ciures are unclear * 

^lici^s for revii?H of stand^ird^ or procfjdures are 
not clears Lha ^Qtne> --as railed to Q^^morwitniu 

^qeriev st^'-^fing is. u*>aae<iuae. e of i^u^^tionn^Je 1 
ler-^a.gf personnei qua! ifirriit iont? or cmf^xlm^ni of 



lack o? attention to standards characteristic of Uio^^ at the tap 
^if the fiaure, WhUe_^m3ny of tht fsvorabhr ^stalements about the 
an^rneies — ^ the center of the figuti* did^noL appcrar fbr^ihase at 
Km^r^P, the fsUc^fin^^ mJditionai statetni^nts i^oul staMards ir^i^re 
impr- often m%iqm6 to agencifi^at the battd^ or figure than to 
thenei in trW caddie Und npt at to ^^ose at the lop), ^ , 



An active prgqram to improve kcrediialiun stan- 
dards or evalufittve mmmu^m is carried out. 

aneney monitors eontinuing cofiipl ianel with 
ot^nciarc^s throuqh roqular reports submitted bv 
ifMitityt ions or proqri^ms* ^ - ' 

InslHytiofm of ibrocira^s 'appro the ^cem^lish-' 



On^ iniUsily r^^^Hng aiip?*ct of the vertical dimensibn is trm 
prifagiiMn«?f^e St us iowr ^nd of riv^ agencies cqgniied f^f only 
^ year i* None of the statements that ehartcterixcd thoso 
agfi^ncles f»s a group was unfovorafaie* If agenciee at high end 



of tho d ir^ens ion ore ^eak and those in the miriuie strong, thoDe 
at the loH ffn:i of ihn dimensiDn should be particularly strong and 
wDiild not ^tpccted to have their ppriad of recoqnition limited 
S^o t>*o ve^ifB, Review of the staff summaries Tor those aqencies 
?ihn^eiJ tr^iit. ail five ^ere strong, ^1 1-establ iShed agencies that 
^ ad prewiou^ly btjijn recoqniz^d for faur years. Mowev^?rt in every 
cMe ^^o^n^?^ deficienc) had I he Advisory Committee to recommend 

rf^Ct*?cl in^ deficiency. BocAusfi* the agencies^ doriciwcies w&xe 
%rfn different ^nd not relotea to standards, impartiality, or 
rc^pc-^^i^enb^a^ the^r lorntjon on thot dimnnsi^n i^as imaffected. 

v^hijp the V er* Lc:^ d imenn ion orders aci!: cies frv-i top to 

batt%3ff5 in rerms of increiisinQ strenath ^tin respect to qeni^fai 

of inl*?grUy. r^spsnsivenpss , and ^oncQvn far £t??ndards» the 

norifnntai dimemjon sc^nsrates. them primarily with respvjct to thf*iT 

i^Ujii! I ve pracedure^ » int«*c|rity. and reputation,^ AqenciM at 

mt' ^^tftmi^E an th0 . her i ^antal di^r^ension re described ' itn 

.f*rqe nki^^^f-r.pf fjivorab-le comments^ ihat ^Iso chHractc^rized thr- 

;;i^lf^ne if^*^ clust^r^d in thi^ center of tha fiqure* Comments about 

^ ij^jnnc urn toward the right includvd, in o^ldit ion, sevornl ^hrit 

* 

refJ^ct^d favarably tWe iraininq' ^ind I'unct ioning , of . v is 
brofiHP^ftceptanc:© in orofeisionoi fields* Aqeneieo on the left 
pfacmiuten^ including inmJ#qiitfto fiit© ,v iaUii * 
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... _ f 

in q^Tietu . the mar e f feet i ve oqenciLrS, as describefl in 

'TAC Btarr rtpbrt^, ^iM^uld appear toward the riqht and bottom in 

riqure 2, with the i is efTective aqencies toward the left anr rhc 

top, Tne oititnbut ion of the dec isions fits that, pattern ^with soiue 

exceptions. Of tt -12 aqencies, that were denied recoumtion, asked 

to shQri rays*?, or recognised for on;, l ne year, eight £U e in an 

arc runninn fron the top to the reft of Iht^ fiqure. lue ^trr qesi 

^i---:-^, ^cwr% 2r = -t!— no )rococ:^\vv•^ r.r fo-.^ .p. ^.e. 

middle of the space rslher than in the lower r.ght quadrant', as 

'hould t\j ^xj^^cteci, fh^ small group of ac^e nc ies wi th two-year 

recognition npfMSinns 'in 'the lo^er portion cf the figurG hm Dccn 

Hrnti^ied as str ^-^q k^encie^ which nevertheless hRn ,ueen -i/dged as 

having def ic lenb le of some kird* The agency with a tour-*ear 

deci^iiah * nward th^ lert side nf the fiqure, snd the a^:sncien with 

aeniTtl and shQ><-cause decisiDns in the center, were anomalies t^ it 

merit p^amininq, , ' 

■=•. 

I^D nc;-ncy granted recognition for four >ears, but \^hich 
nevertnelesn ' ;^ enrs uncharacter ist icsl i^to the left of the mom 
grnup uf ciqenciM, \n described in the staff report 'With a mrher 
of nt n^_f^irnr^f^ roncerning U s organ uat ional structure and aci red 
ing prncedures. AUhouQh ravorable, the ataLPfr^ents neverthel 
tended to he -n--. in'ad wit^ aqcncica laward thfc Idft cir^d uf \ht^ 
hori^ontai dimension. !^ unfavorable statements, alio associated 
with the Ivt side of the fiqurf^, concerned vagueness in tt'^ 



state^^*!?f . ■■ tvln^^so to ^valuiitwn prQCedureG, the cmimnt mpv^t 
of the hcr.,.w.iiai dii^^ension, w;i?re favoraJ lii and similar to thost^ 
fnndt^ ^ibou^ t^?*^iyrq^ .:??ritfai Qmrn? at mcncw^, Pr^yim^ly reeon- 
ni^vv! for two f^mt^t th^ ^iClflnry'A rour-yijar r^coynUi i reco^-- 
jT'f^nd -d bv the Advisfiry Cammittc?^, **ith a reques^t ^or. i^^? jntefi^ 
report an th^ uqi^^'c^'n proqrec»a in clarifying iti^ appeal and 
complain* DrDced in this ca&t?,- th^ c^yency^s pos^tioTi io t^^*- 

nf^nts not related to the dormant anpec:' af hori^csntal d^n^en- 
iS40n, pTun two unravorabie statements that did not affect the 

The t«o aqtmcies that w; * jud^M to be loir^ in.tH«ir ^ibility 
to net^fffme ^durBtioniit qusMiv**^© den:er| rfcognitiQn and iHn 
o!iher 415*^ to ^^nq^r c^us^ *^*iy its recqgnUion should not he? witH^r 
dr i*in--prri\ i fi^d^in interefit inq oontraRt, fhi^ ^qunc^y denied r* "oQ- 
ition was a case in s^ucH Ihe decision wnu c^ues^ ianeble. If Uic* 
\*pri^inn «nn intic^c**^ rorr*?ct l.H^*f^ ^^brr s'^^or^otion on w^^^ltiM it 

^ uaned did not get ex. .tossed cl^^iifly in the DlAi: nt^ff rt^port * 

fHht wC?ei!iion ^mpiy suppoivled by slati?m0ntn art the stnff 

reporti ti e frsiti ubb mi sensitive to thm* 

fhf* staff repjft on the iqency derued rfjo<iCjnit tan c^ont5i.m>d 
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sO sppurate statements listed in the cataiog of S59 slatements 
durivtid Tom iiic eariiHr reading of starf reports. Of eKfise, M 
"Cre iiLaler.enta aiso characteristic of th:? highly 'regarded agencies 
clustfi-en in U;e miadle of F :r,.y« Ail but thrse of the iromain- 
ma 3C «tre eirher favcrabls -itatemenrs ^e.q. , on-site review teams 
are s-rnt rRievant -material about the school well in adyance of 
the rewr-w; of noutrai oneii (p.g., thft agency does nut use i 
pre .ccredilation statuaK Iwo of the three unrB^or able al/atements 
<!'*a!r w;th fairly innDCUCUE issues Icq,, Bomo or alH of the 
meetjnns to review the visit inq team's report have yeen held 
without a vusitinq team bember present ) . The third concerred 
?»ff!bjnuity or lack of completenpss in the agency's standn;rds, a 
'■'"rfy ESTi3us dtrpcicno, but .ne nc;?, .msuwI 1 y sufficaent to 
■ji tlfv deniii.' the ,ac]enc i es appearing in thg lower part of t he 
f, w£« «» t^' , tHft-yea'r decis.ons had a qreQter number o^.'eqijallv 



Ine Adv isory Cbmm ittee's recommendat ion' tor denisl of recog- 
nttion to fHfl 3r|enc y wh i ch was less than five vears old ana 
pr^tit.oninq for reconnition for the first "-ime^ plated that the 
ni5<»rfcy failed f g demonslrstf compliance with t»rts ct;Jteriai that 

IhC; agency nerve "rlpariy idefit i find' needs," and t^ot the mem, be 

■ -■'*■' "I ' ■ - ' ' ' 

"niitionai or req tonal in As scope of operatici^, " Neither of 
iHestf faaures was nou'ri/lrt' the stsff report'; and ^^ffirBuse the 
eqency had a Inrqe num6«r dr qualities similar to, those of 6^1 1- 
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reqaf deF^gencies^,~R Appear TrT t he 
multidimensional scaling analysis* The decision to deny recog- 
nition was withdrawn .less than six months after it was made, 
and recognition wae granted for one year, 

,^_.._^._.l^.ig^5.y_Ahat asked to; show cause why jts pi 

recognition should not be withdrawn was described in the DEAE 
stafp^ report with 8 5 statements, 43 of them characteristic of the 
.effective group of agencies clustered in the center of the figure* 
Of the other ^42 statements^ bLx described fairly severe'^^icien- 
ciesj such as jpossible conflict of interest, and limited acceptanfre^. 
in Its'.profassional nield. The staff report showed substantial 
justi ficfition for the Show Cause decision. The large number of 
representative, favorable statements, and the scattered nature of 
the deficiencies, with only one or two of the unfavorable state- 
ments r^iated to the two dimensions of Figure 2, resulted in the 
locat^ion of an agency of questionable merit among the more repre= 
sentative agencies . ' ' \ 

The results of a similar anal >^s it. with the other random half 
o f thegcegional and special ized agencies dn which decisions were 
TTiade during the pBrlod - from 1975 ih 1978 are shown in Figure 3. 
Here five agencies are widely separated from the others, with three 
' or four more somewhat detached from a^ central group that his been 
forced closer together than the agenclis\ shown in Figure 2 in. order 
to accommodate the. marked* dif feren:ges of the outlying^agencies 
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Figure 3* Location of recnqnition riecisinnH in two- cfimenslone 
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With one except ion^-the agency granted recognition for two years-* 
the outlying agencies were either denied recognition ot withdrew 
the petition for recognition. 

The horizontal dimension of Figure 3 ia much the same as the 

vert ical dimension of Figure 2. Eighteen Btstements described 

. . 

4 

common aspects of the two dimensionp. Agencies located toward the 
fight sid6-of Figure 3 were described with statements- critical of 
their integrity, ttteir responsiveness to the- interests of their 
constituencies and the general public, and some of their evaluative 
procedures. Agef*fsa,es toward the left were judged as giving close 
attention, to evaluative procedures^ such as providing training for 
visiting teams ang encouraging their accredited programs to engage 
in continuing self-evaluation* In addition to its close similarity 
to the vertical dimension of FigUie 2, this dimensiDn also shows 
some of the characteristics of uhe horizontal dimension of Figure 
, 2.^ In particular, some of the attention to evaluative procedures 

„. ;that^ d ist inquished_betwe^ .at..:the^tight _and^J^.ft sides of.-. - 

Figure 2 "is apparent in the distinctions betwan agencies at the 
right and left sides of Figure 3 as well. Thus the two Fairly 
"distinct dimensions of Figure 2 seem to have been cbllapsed into 
one dimension in Figure 3. 

The vertical dimension of Figure J is^difficult to interpret* 

p= - ■ ^ ■ ' 

The agencies lying toward the top and bottom extremes were not 

consistently di f ferentiated^by similar statements* The two agen- 

o 

ERIC 
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cles in Figure 3 that depart from the pattern of strong agencies at 
the center and weak agencies BrDund' the, periphery are the agency 
recognized for two years in th- b'or-? upper right and the agency 
.recognized for four years ab vv ■ .■ir^; slightly to the right of the* 
main group. These two would ao.-. ;, : closely to the pattern of 
the other 41 agencies if the first ... ienied recognition- and the 
second wore racognized fnr one or two „M's, 

Seven statements applied to the agency in the upper right 
'corner' of Fiqure 3, which was graritrtj recognition: for two years, 
were . also applied to .-ther agencies located on the right side of 
Figure 3. T'ley were ralated to- ethical standardsi the Impartiarity 
o f decisions I the represantativeness of policy or decision-making 
bodies, and the preience or absence of public representatives on 
these bodies! and the adequacy o f eel f-?; tudy procedures and the 
-agencyJ-s^ own -evaluat ion- o F its standards These sMis i^^ 
all three of the types of criteria reflected in the horizontal 
dimension of Figure 3 — integrity, responsiveness i and evaluative 
procedures,^ . ^ = . • " 

The OEAE staff recommendation to the Advisory Committee was 
for rtfcognition for one year. The Advisory Committee^e recom- 
mendation to the Commissioner was for recognition for two years, 

with an interim report in one year that would address the defi- 

* 

cienciea notetf above. In this instance the agency was similar to/' 
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other agencies denied recognition but was treated les-s severely. 
One reason for its mpre lenient treatment was that of the five 
agencies farthest to the right in Figure 3. all except this agency 
were applying for initial recognition^ and three of the four had 
been operating for less than five years* This, agency had been 
operating for more than 20 years and was applying for renewal after 
an earlier recognition period bf three years* ^ 

The agency^ with the four-yesr recognition decision farthest 
toward the top in Figure 3 was evaluated under the 1969 criteria. 
The inclusion of a number of statements in its staff report that 
were more common to' the 1969 than to the 1574 criteria moyed it 
away from the main group of agencies* Its only djeficiencies--in 
the area of self-study ptocediljr es--were.^^crtedi 4n the Advisory 
Committee*s recommendatlbn . They were not substantial enough, ' 



howeverj to, move the agency very far toward the. right in Figure 3^ 
or to cause the Advisory Committee to limit its period of recog- 
nition, 

summi p 

Locating agencies in a two-dimensional space in accordance 
with the differences among them revealed consistent ' patterns 
of characteristics associated with the different recognition 
decisions* One dimension is related to the agencies* impartiality. 
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feeponfeiveness to various constituencies as well^ as to the general 
publict end attention to the dey elopment and application of 
■ standards. Agencies strqng in one of these areas tended to be 
strong in the others, Vhe other dimension separa^tes agencies 
according to the strength jf their evaluative procedures and the 
degree of acceptance accorded them in their particular professional 
field, ^ / 

An agency's location in this two-dimensional space carries a 
fair amount of the information on which the renognltion decisions 
were based* ^ As an agency's distance from the central group 
increases, either upward or to the riqht in Figffre 2^ the agency 
is increasingry likely to show def icienibies .ssbciated with the 

I - ' ' 

two dimensions just described. That the agencies actually denied 
racpgnitlon or recognized for only one yeflr tended to be the ones 
farthest removed, from the central group indicates some degree of 
reliability and validity in the decision procpss. It is reliable — 
to the extent that the agencier^ recotinizocl for four vears are 
^ located in a common, limited ragion of the space ^11* the agencies ^ 
denie^ or given limited recoLgnition* are'consistently separated from^ 
^hit region. If the process were not reliable^ the various types 
of decisions would be intermingled throughout the spac^, Jhe 
validity of the decision process is indicated in the nature 
of the agencies at the extremes of the two d imensibns *^ Sub- 
stantive deficiencies with respett to a number of the criteria ore 
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associf ited wl th the regions where ^hhe agencies rienied or given 
iimittd recognition are found. If the decision process were not 
valid-^that is, did not permit inferences that the agencies denied 
ptcognition were less, abple to provide judgments of educational 
quality than were the agencies recognized for three or four years-- 
the, two dimensions would not reflect the presence or absence of 
substantive deficiencies , 

Clearlyj the decisibn process has some degree of reliability 
and validity. Equally clearly^ the analysis Just described does 
not provide all the information . necessary for more complete con- 
clusions about the vaJ'idity and reliability of the recognition 
process. Questions can be raised about the adequacy of both the 
data and the analysia. 

The data were limited to the content of the DEAE staff 
reports^ which may not have reflected an agency -s characteristics 
entirely accurately* If they had been completely accurate, the 
analysis used would still have ignored characteristics that 
were rarely reported, but which nevertheless may have been highly 
important in any particular deci&ion, A partial check on the 
accuracy of the DEAE s* ff reports was provided through an indepen- 
dent ETS staff analysis of a sample of the original petitiGns 
submitted by agenclis. This procedure and analysis will be 
described in Chapter VII, 
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The .loss of information about rare but important character- 
iatics of agencies results in an inevitable but not serious 
reduction in the accuracy with which validity is assBSsed. 
Validitv applies to the Justification fnr inferences drawn from 
thr prtvcess in general i it is not applied separately to indi- 
vidual cases . The occasional decisions that occur in unusual, 
circumstances, and aj a Result depart .from- the' general pattern, 
either properly or improperly, do not affect the merit of the 
general r'un of inferences dr%wn from the decision process. The 
pattern of the decisions is the proper basis For judgmentia of both 
reriability and 'validity. ' , / 

;Some limited evidence that the recognition process is both 
reliable and valid was found in the ways consistent discriminations 
were f^ade a"'ong agencies jn terms of the extent to which they, 
satisfied the general body of recDgnition criteria. " More extensiv/ 
observations related to reliability and validity will be reported 
in the following chapters. 



VI. IMPQRTANCE OF AC&NCY CHARACTERIST IGS 

^ . i = . ^ 

If recognition decisions are to differentiate agencies that 
provide sound judgments of educational qyality froffi those that do 
notj the observed differences among agencies must reflect their 
^ different levels of ptrformance. Other kinds of distinctions, 
however useful for some other purpose---as j for example, with 
respect to the provision of other professional serviGea--are not 
partinent to the validity of recognition decisions. A major task 
of the present stud), therefore, was tv determine what persons^^o 
view accrediting from different perspectives regard as the /most 
Important attributeg of accrediting agencies in their role as 
tvaluators of educational quality. One perspective" was provided by 
the executive directors; or chief administrative officers of accred- 
iting and state approval agencies. The directors of every agency 
that had ever inquired about recognition by the U.S. Office of 
Education, whether or not a formal petition was eventually subm^it- 
ted and whether or not the agency was subsequently reGognlzed, were 
inyited to contribute their views to the study. Others were added 
frJfm the list of /agencies recogniied by the Council on Postsecond- 
ary Accreditation .(COPA) a/id from the membership of the CoLncll of 
Speciiliied Accrediting Agencies (CSAA). Correspondence with the 



agencies indicated that some mr& no longer functioning as accred- 
iting agencies and that otHerSt after ^t^avir^ madft initial inquir- 
ies to DEAEj had never taken on an accrediting functiorf* Ihes^ 
agencies were excluded, leaving a final list of 129 accrediting and- 
state approval agencies. Some oF/ the character ist Ics of those 
agencies Sre shown in Table 5. . 

TABLE 5 ■ 

Number of Accrediting and State Approval 
Agencies of Different Types 



Regional 

Nonmed ical 
Med ical 



Rtcognized Recognized Total 



IT 
0 



0 

• 0 



11 

•0 



state. 



Nonmedical 13 
(Vocational ) 

Medical - 7 
(Nursing) 



18 
10 



Special ized 

• Nonmedical 
Medicfal 



23 
28 



28 
11 



51 

39 



Total 



82 



47 



129 
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Obtaining the judgments of Meads of accrediting agencies had 
thfoe purposes. One was simply rn'jrtpntifj, the degree of" ispDr- 
tanee they aU^cfhed to attributes of such aqencies in the evalua- 
tion or edueatianal quHlity. rhe second was to determine whether 
^thRir views difrered, with the type of agency they dirtcted. Ihe 
third was to 'prov ide \an author! t nt iv^e set of judgments against 
Which the critical qualitiei in the recognUion decisionB could be 
evaluated. If attributes considered important by the agency dir- 
eetOTS were neglected in ths recognition decisions while others 
they thought less ifrportarlt played critical' roles, one aspect of 
the yaiidity of the decisions would be damaged. 



The 44 criteria themselves seemed too diverse in their' 
■ - ." / \ ■ , 

specificity-jtsometimes very broad and sometimes quite narrow— to 

permit interpretable Judgments of Iftiportance.. They h^d also been, 

thoroughly discussed during their formulation and were likely t& 

•elicit automatic, wel Urehearaed judgments. A selected set of 

■the descriptive statements derived for each criterion from the 

DEAL staFf reports was thererore ustd, since Judgments of their 

Importance were likely, to be direeted |q the content of the itate^ 

mant rather than to the complex web of associations each faniiliar 

criterion v^uld evoke;/ * >^ 

' ' = • • \ ' ' ' . ^ 

Because, soliciting judgments on each of the 559 atatements 

■ \ 

was .impractical, some procedure vras needed' for selecting a compara- 
tively small number of statements to be judged. A ser|es cf factor 
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analyses oT the statefnonts that appeared in the first group of 

,-. ■ 

S2 staff reports haw lueMt i f A£d Sevsn factor &--that is, seven 
moderately independent groups of mutually related statemtnts* The 
52 staff reports wtre a stratified random samp ie of tht reports, so 
thos^ factors could be ejcpected to include the most discriminating 
stattments in the full Bet of reporti. Accordingly,. 80 stalements 
werte selected to represent those severi preliminary factors, with 
■^a secondary stipulit ion that at least one statement for each 
criterion ba included. Only Criterion c3f that the agencv have 
least two years of experience, was not represented* Ihe seven 
^dtly factors were, as expeptedi similar In content to the factors 

^ ■• , ' ^ ■ I . ■ ■ ^ 

described in Chapter IV, but organiiid somevs+iat differently. The 
similarity was great enough that each of the nine factors based un' 
the complete data was represented in the' 80 ststements, , ^ 

AGENCY DIRECTORS' JUDGMENTS OP IMRORTANCE^^ ^ 

The .80 statements were sent in the fall Of 1978 to the chief 
administrative officers o^ the 129 identified accrediting and state 
approval agencies* They were ^sked to ji Jge the importance of each 
characttiristic described by the statem^i^nts in de'termining the 
quality of education offered by an institution o/ program* The 
judgments applied to a somewhat specific functiQn--the determina- 
tion of educationol quality—of a spfcific agehcy, with an option 
for the agency head to report ttiat ihy particular statc*ment was 
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^not pertiWent to his or her agency's* opefgyon. The daUrrnination 

of educational quality was not fufther^^^ it wis also 

W ft und&nnBd m the iagislition proyidih^^ 

liHt of rtcoqniitd sgencies, a mora specific definition miflht have 

1 1 - il p d ih& scope of the judgm^ts morl^^^M fry'. 



iHf^ 'jydqmervts of iR^poctinct we r0 recorded OH a siv^n-p ' 
^rriU f^ram no iir^portancfe to hlgh^ ifnporianca 

pr*?r.*'ntc-d m AppendiK withHhe stetements organizid^^ to 

thp^ofiq^l seven ifacteorit as they were preainted ln^tha 

n^ire. 1h© Statements in Table 4, are those ^t^ 

-^c-t rrt quently judged hlglg^in" impQf tance,^^itK ratings.oF ifi orr7 

uy ^ij p&vQent or ^ore ^ the f 2 agency heads^^ 
t i^ie to have their .jud^itBnts eounted,^^^ Those 92 repr^eented 71 per- 
fpnt oF the identified aceridi ting -Wd e^^ 

^^epfpsf ntat ion ory^fr^rent types of agertclM.rer^jed Iffcim 62 to 92^'^ 
; ' r aont , )P<3 th. the: unrecogniMd ag^nciee hfving ;th^:;pqprest-:rep^e^ e 

the most Important Qualities of aD^editlng^^^enc f 
i^^v \ of their direc^oM:f were goo^, evaluative procedure^^^ 
Statements 240, 272, 2^1, and 128), ,f J^oroi^.,but TO ip]piicatlon 
of St nndards (Statements 55, 593, 59^^^^ 



--^..^;Stetiments Most Frequently Dudged 
oT"^M#gb Impdrtance by Agency Directors 



. / ' I \ „ ^ ' •Percerifcagas 

: ''\ \ ' " '/ dF^ Ratings 

Crlterian ; Ititement ^ * ^ 1 to 5 6 7 

. aJiiiA 240 A ielfrstudy ar\d/Qn-slte review " 4^ 7 89 

. ' ' are required th'mt provide for , 

> ^ ^ the quaiitativt isseasment'of 

an instlt,^fridna'B or. program \s . r 

\ ^ ^ -/^ strtngfrKs and wt^ . ____ ■ . . . 

b2iiA 55 T^it agency publishes^ the standards 4 10 85 

■i^y which institutipni or programs I 
=;are esvaluit^d, . ; ^ - ; I 




■b3ii : ^72 Visiting' teami engage in discussions ^ V 

/ with racultyi : administrators J a ^ 

- 'Students. 

b3iii . 261 £ach inititution or program is given 4 .17 79 

\ : a copy of tht visiting\ ttam'i report r . . ■ 

. ' . V which Indicates strtength^t ^eaknessea, \ 

I A and recornmendatdohS' for^imprb^ 

b3vii= 470 The agency reports, the specific. / 4 22. 74 
reaioni for adverse decisions to ^' . 
^ the chief executlva officer .of thfe.^ 

instiiution or program and offers 
_ an* opportunity' td appeal * ^| v- ^ 

^b3vi 460 Accreditation is withdrawn Q^ly $ 19 76 

' after due proc'SiSi for cag^e, and . 

with ^due notice* ; ; : ^ * \ , ; 

Jb8 ^ 591 Wrlttari procedures clearly assure ^ 6 22 .72 
^ that only . qualified institcitlons or ' 
\^ progrsrps^-are accredited or approved, 

b8 ?d9l The agency's decisions .reflect fair L 17 77 
. and .conslitent applyicatlon' of its 
standards* 



Criterion 



■ ■■ TABLE 6 ■ • 
(cont'd). 

Statements' Most Frequently Judged . 
of High Importance by Agency Directors 



Statement 



. Percentages 
of Ritlnqs 

.1 ,to 5 6 1 



a3iiiB ^ 128 Written inetTuctionai guidance, 
and consultation are provided 
" ^^or^ tjiie ; sel;f^tady and^^^^i^^ ; 

' ^ Visit r^li; ^. . 

b9 54 Institutions or programa are 

vregularly reylewed^at reaeonabiie 
. . Intervals * - ■ 



7 18 75 



7 21 72 



b8 590 Th^ agency Vs^ published procedures 

_.^ .._are fairly ippliacff ' its 'actions . 
^ are consistent with its procedures. 

blO^^ 651 Institutions or pfograms are 

' \ ^cleatiy^in formed of the jTtturer^^ 

and scoplT^f~th%aT~a~ccTedltati 
^ ' status* ; 



8 12 80 



b9 



e31 



.bli 



598 When circumstancei warraht it, 
institutions or- programs are re- 
. evaluated at intervals shorter 
than the normal cycle. 

40 Prqcedufes for granting, denying, 
withdrawing, or reinstating 
.accredited status are, clearly 

' descr^ecl* ^ 

270 Faculty members are given oppor- 

/ tunities to make their vlev^ known 
/ to the agency. 



9 25 66 



10 . 18 ' 72 
10 31 ' 1^9 



/ 
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■ statements were relsted to the sgency's organizetionil scope and^ 
structure (e.g., the scope of the agency -s, activities ii neither 
too narrow ^nor too broadp fees ere 'reasonable) arid the; agency ''s 

\ responsiveness to its constituencies (e*g*i all interested groups' 
v^re represented on the governing boardi accreditation status and 
th'e dates of next review are published). These differences were 
not gregt i -howeviri slnc4 even the statements judged comparatively 
low were still important . All but 11. of the 80 statements were 

.judged liirflftr than "Of Moderate Importance" by at least half of the 
directors* the comparative nature of the differences in importance 
must therefore be stressed* NeverthfelesSf the qualities most often 
judged high in importance were thoie associated with evaluative 
procedures 1 fairness , and due process* , 

Importance of General ^gu8j.ities , ; 

The" seven groups bf related statements into which the 80 were) 
organized were each given a label, were described in one or two" 
phrases I and werfe^ presented to the directors , to be ranked in order 
of importance. In these compar&tive rankings, the broader issu^^s 
to which the statenients refer- were judged rather than the partic- 



ular characteristics that contribute to the . evaluation of /in 
agency. The variation in importance of different specific char/ac- 
teristics assoaiated with the same more general quality wa^ oier- 
ridden in these judgments. For 'example, as is shown in Table 7, 



the agency's attintion to its standards waa most o^ften Judged the 
most impQrtant quality of the seven* Yet some detailed aspects of 
that attention to standarda-^.that they are validated against the 
performance of graduates of accredited prograiiis, and that informa- 
tion on standards Is exchanged with other agencies--were individ- 
ually judged quite low in importance; Other statements related to 
standards-^hat their validity ond reliability are evaluated, and 
^that cdntinuing eompliance is mbnitored through regular reports-- 
ware judged aboyt in the middle of the -80 statements, which gave 
them somewhat more than "moderate importance." Still others—that 
standards are publishrd, -and that theyVare fairly appliea—were 
quite high. 

The agency directors were quite clear In identifying the two 
most important/of the general qualities of iccredltlng agency 
^performance listed in Table 7, Attention to standards was judged 
first in importintfe^by 40 percent of the directors and second^ by 
over ;3d percent more* The quality of the evrlyative procedures 
was a cle^r choice for second placfe. The- other flvW vfere far 
behind, . ^ ' 

Again, the qualities least often judged highest in importance 
were not unimportant; as was indicated by the judgments of high ; 
importance fot' some of their contributing statem^entSi Due Process, ' 
for example, was placed leaSt often' in one of the top two broad 
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TABLE 7/ 



Comparative Importance of General Qualities 



'Pefcant Percent 



/ 



Ranking Ranking First AvGrage; 
First or Second ^ \^ ^ ■ Index oF^^ 

(N^92) (N-92) / General Qualities ' ' Importance 

40 73 / / ^ Standards i The clarity and- . , 75*6 ; 

appropriatenees of the.age 
standards and thii qar^ mt^^^ 
it develops and maintains them. 



\ - 

\ 



27 'z"^/ 56 Evflluative Prqceduresi The com- 81*4 



\^ /: ^ prehensiveness and care with 

' v^ich Instttutions or programs v;^ 

/ 7 % are^evaluatwd* , ■\ 

\m/ ^ 23 Agency Inteqrity i. The degree of ^ 73.3 

^ / ' ' trust angendered by^the agency 

^/ \ through its reliabiltty I autonomy^ 

/ \ and concern for. ethical prActice. 

20 ^ Accrediting Procedures i The 7S.5 
spiclflcity of th^ agency ^s pro- 
cedures and their pertinence to 
its purposes* . | " ^ 

14 > Scope and Organizatlon g The 71.5 
clarity of t,tii , agency 's purposes, - 
: the effsctlvenesf of- its orgahizs- 
tional structure^ :and the adequacy 
of its reaourc^tes. . 

' ' . jK .. ' ' , ' . 

8 ^ Responsiveness I The sensitivity • 67.8 
^o f" the. agency to . the heeds and 
.desires^ of tKli groups it affects, 
and its readiness to respond to 
them* , ^ ' - ' - . >r 

7 : Due Process I The care with .which , "84*7 
procedures are ^ structured and ^ ' 
codified' to avoidUnfair or 
capricious actions i ^ 



®See Footnote 2, page 103. 
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quilities^ Yet two- of its components--withdrawing accreditatiori ' 

only ^or cause and after due process, and reporting specific 

■ ^ ;■ / ■ ■s'.- ■ \ ■■■ . ■ 

refjsona for adverse tiec isions—vyere given the highest rating of' 

importance by 76 and 74 pLrcent of the directors', respectively. In 

fact, .if the ratings.Wf the individual statements in each group ire ^ 

averaged. Due ProceW emerges as highest in importance, with , 

Evalu^ation Procedures Wlose behind. Standards is in fourth place--.' 

behind Accrediting Procidures, despite its. clear preeminence in the^ 

J udgmints applied directly to the general qualities. These average 

.{'Judgments are shown in the last column of Table 7.2. 

Jhe discrepancies between the Tmportance given Due Process 
and Standards when they were judged as qeneral qualities and when 
their standing was based/on the average ..importince of their compo- 
nent statements merit oxamination. pueprbcess. In particular. 



1 ■* ' • " i» ■■ ■ I ■ 

The Judged importance of the' statements averaged toestimate 
the importance of tfie criteriV was defined aa the sum of the 
percentages o f ' 6 ' and, ' 7 ' Judgments less the sum/of the per- 
centagei of ' 1 ' and fZ' Judgments on the seven-point' scale of ' 
importance. For. e)<ample, ■Statement 572 was given Judgments 
ofJ7', '6', 'as and M' by 45, 24, 3, and 1 percent of the 
direotors. Adding 45 and 24i, and subtracting 3 and 1 give its 
Index of importance as 65^. " jh is index was used rather than the 
median or mean values to translate, the disbritiutions of ratings 
into a v'S ingle nymber to represent lmportance\« Medians would not 
.give suf ficient weight to the gmall but- impdrtant numbef of low 
^ Judgments' in the a-shaped distiiibutions of imbortancej^ and mtans 
, would glva .them too much, freight; • The shape of tha di^ifibutibna 
, index desprlbed. more fnUrpretable 'than either of 'the 

:fflOPBKo,p'iiJhon .measure central tendency., The.values of the 
V index.: ranged from 16 to 96, with a median of 70.5 and- a mean of 



' . • . ■ ■ ' ■ ■ , . .J • :■ ' 



is puzzling in its shift from last place in judged Importance as s 

general quality to first place in the iverage importance of iti 

related itatementi,. Th t likely reaeons are'either a fiilurt 

of the statemehts to rtprestfit the general quality or a failure of 

the definition of^ the general quality to convey the meaning of /iti 

related atatementa. ■ '/ 

A ■ ' y ■ ' ■ ^ 

: : . The at atem^ pue^&i^obesi ire <shpwn in^ Table 

8. Eight of the ten were judged quite high in /rniiQrta The 
definition of Due Prpces8;^&i a general; quality tre^s it as a iset 

of formal ppocedurea to prevent unfair or capriclouB actions. The 

■ . ' ''^-^ ■ ' ; ■ ■ ■ ■ " ^ - -^i ^ ■ ■ ' " 

existence of pppeal procedures j a requirement that withdrawal of 

iccreditat ion be on]y for causey and other procedural prdtectlona 

y mayvnot be adequately conveyed by a phrase limited"to~the=avoidance 

of unfair or capricious aotionB. To that e^ctent, Due Process is a 

more important quality of accfediting agencies than' is indicated by 

the judgmehts reported in Table 7, ; . 

Standarde--the care and attention they are given--ahows a 
discrepancy in :the other direction. While it was the moat impor- 
tant general quality in the coilective vie<^ of the agency dlr-, 
ectora.j it fell in the middle in%terms of the average importance 
l of ita related statements, which are^ listed in Table 9. / The 
first six . atatementsiJ in Table 9 were all atfove the median value 
i^in judged impprtance,.^ -The other five pulled the averagi* down . 
^.for these statements, but Due Process, Evaluation p/rocedures. 
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TABLE 8 

■■ j . ■ ■ 

Importince of Statements RBlafed'to Due Prdcesi 

. r ■ ' =■ \ / ■ ^ ' ' ' ' . 

. - , N '/ Inde}^ of 

^ ^Statfement * Importance® 

470. The .agency reports the spiclfic reasoni ^or ? 
adv^erse declaiona to the^chief executive 
' of ficer of the ihstltut/icn or prograni and . 
. offers an opportunity jto' appeal* ^ 95 

460. Accreditation is withdrawn only after due 

" ; process ^ for Muse , "^nd~wi^ 9^ 



590. 



The agency's, published proceduffes are 
/fairly appliedf Its^actibns are consistent . 
< with its procedures/ ' . - ^ 92 



441. The chief BKeqi^lvs officer of the ev^^ 

^instttution dr^^cogram is givsn^^an opportunity >^ 
i _ to respond toythe visiting tesm report. 87 

510. Notification/of specific reasons for an ad- 
verse' act^h on arf appeal is required. 83 

471. The agenc^y has published rulei of proce- 

- dure regard ing^appeals. ,78 :; 

,500. i Adverse^ decisions are follbHed by, a hear- 

. ■ . Ingi at which ^r^resehtat Ives of thfe * 
. affec'ted institution' or ^prograrn may appear . .75- 

^ " 7^ - . \ ; ■■ . . ...."» . 

490. An^dnstitution 's or program's accreditation 
sbetus cannot be chinfed by the agency ^\ 
^hile an appeal, is pending. - ^ ■ 74 

484, i/Appeal proceduresr include safeguards against ^ 
A bias I e.g,, no one in\^olved in the or.lginal 
/ decision may serve on the appeal committee. . 59 

^ -■ / . ■ r r ' \ ■ ■ ' ■ / , / > / ■ 

450.' "; A^t least one member of the visiting team is ) ^ 

^' / . required to be present ati: the evaluation of 

/ the visiting; team re^rt. \ / ^ * M 

a - * ■ ' ' ' * ' * " « 

Sit Fdotnpte' 2, page t03,' . , : / 
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Accrediting Procedures^ and Integrity all come out higher, than 
Standardi if only their si^ statefflentB judgid' most important are 
averaged* The most ^.reasonable conclusion vseems to be that the 
general quality related to the attention an agency gives to its 
standards is not adequately accounted fo^ by the specific observa= 



tions. related to standaids that enter,^*^e recognition process 




55, 



411 



380. 



55i2, 



556. 



•382. 



. _ : JABLEL? . . .J.^- 

Importance of Statoments Related to Standards 

Slatement . \ 



The agency publishes the standards by 
which institutions or programs are . ^ 
^ evaluated. 

'\ '* ' ' ^ * ^ . 

The agency's standards are clear and 
detailed enough to assure fair and ^ . 
reliable treatments to institutions or \ 
prbgrams * _ • . ' \ , 

Standards are not revised until after 
interested groups have had In opportunity 
to react to the' proposed revisions. 

^ ilk . ■ ...... « . , ' 

vThe agifjcy regularly revlev^ its standardsj; 
policies |. and procedures by inviting sugges- 
tions from various iriter.este^ ,grpups. 

The agency monitors educational ttandards 
to keep them consistent with occupational 
trends .v ... ' -"^^ ' - ' \: 

An active^ program to improve accreditation 
standards or evaluative procedures (e.g * i 
studies, conferences^ workshops) is carried 
out . 

Institutions or programs are evaluated with . 
respect to.fheir own purposes rather than 
in comparison with others. / 



IndeK of 

Impbrjtance^ 

95 



:B7 



B1 



76 



72' 




^ . ■//. 



' .. 107 , 



TABLE 9 ' 

" . . ' ^(cont'd) ^ .. .. . - 

Importance of Stitemorits Related to Standards 

IndeK of 

■ Statement Importance^ 

550* The agency tvaluatea the validity and .reiia= 66 
blllty of its own educational standards. . 

The agency monitors continuing compliance 64 
with standards through regular repprtte sub- • ^ 

555. . Standards are. validated- agains 29 
^ ^ accredited institu- 

tiohsy'^ such as. In licensure eKaminationB. ^ - \ 

662. Information rigardlng standards and guide- . 18 
/lines for condyct of.site visits is exchanged 
with ^'other accrediting agencies, : 

a - " ' 

See Footnote 2, page 103. 



These two sets of dlscrepancies--for Due Process an'd 
StandaFd8--betwBen the average importance ^of groups of related 
stratements and the importance of the general qualdty to which 
they contribute--|ugges^t^^ need for clearer conceptualiiation 
of some of the criteriai. If important details are missing from^ 

^he dbservations of agencies cqncerned with, their attention to 
-^aij4dLar^ At the. least, both Due 
Proaess anid Standards need |detilled^^eramihation^-^^^^ 
specific agency attrlbutea and performances that should be observed 

=ln the evaluatipn of those two general -qualities. 
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Importaneg ofi thm Criteria 



The s.tatements Judged by the agency directors identified 

• . / ' - !• j - . _ , ^ • . . • :^ .'^ \ 

specific aqency chirtctaristici related to the criteria. The eev^en 
general qualities that the direators judged were broader in scope 
thanthe criteria, and the two sets of Judgmente were in tome" 
respects not consistent . The judged importance of the criteria^ : 
Since they are explicit itatements of thermal i ties oh^^ 
recognition res'tSi may cla the irtconiistencies. 

Thirty of the 44 criteria were reatate'd completely by one 
or a comb i>iationo restatements. For examplL Criter ion b2iiA 



states I "The agency or association publishes or otherwise makes 
publicly available the standards b\y which institutions ^or programs 
are evaluated ," Statement 55 . rea'^S j "The agenc^ publishes the 
standards by which institutidns or pVpgrama are svalbated*" That 
single statement indicates complisncB with that critterlonj even 
though amplifying statements may be add^d. The judged importance 

as 



32iiA 



.^-p?i^f statement ;55 can therefore be appliib to Criterion 
^ well, ^ When the criterion was more^ complex, Vhe ratings of ^veral 
. statements, werevaviertgeid to determine its im^rtance. The lic^els 
of lmport;*ance of the 30 crite/ia for which j\|^gmenta of di^ct 
restatements, were available appear in Table 10. 
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TABLE 10 



Importance. Qf the Criteria 



^ ' . \ Index of 

Criterion (abbrevisted) Importanee 

Visltlnjg' teams meet with ficulty, 96 
admlnlstratorSi" and etudente* 

■ ' " /\\. , ^ 

A written report of the visit ie \ 96 

provided* . i ^ 

Stindardi are. publ ished. ^ 95 

Agencies may appeal and retsons ' . --95 \ 

are given for adverse decisions* . \ 

Aecreditatlon is withdrawnXohly for 94 ' 

\ eause and after due prDcessH 

?eeyaluat ion oceurs at reasonable 94 
Intervals* ^ * , 

juidanee is "provided ^r the self- '93 i 

study and site viiit* - . ' 



Standards are fairly appliedi 91.2 
deeisiofis are impartial. 

The chief e^^ecutlve of ficer (CEO) . : 87 

may certimenfe on and supplement the 

visiting te^um's repc^rt* . ^ ' \ ' 

"Aeerediting proeedur4s arf. clearly 83,5 
jdefihedi ,/ . . • ■ 

No iPgnetTon Inte / ^ S2 

independence 'of judgment. 

Reactions are^ solicited to proposed ^ 81 
revisions. of standarda* 

Procedures exist to guard against 78 
conflict bt'- interest . ^ ' * 



. TABLE 10 

(Cont'd) 

Impor tance of tht Cti ter la 



b^viil 

blii 

bZiv 

a2v 

'bme 

aZii 

Wi . 
b7 

b21i£ 
a2iii 
bliiC 

c4 



Criteridn (abbreviated) 

Appeal procedures ate provided . 

Purppiea are- elearly. defined. 

Written procedures provide for 
equitable review of complaints. 

Vijsi ting teams, l^cJude one^ meitiber 
not part of the ageiicy* *^ 

Decision 'prQcedyreff; ape, puBli^hed> 

The aqenc )f * a pbi lories pr oqedup^s 



Finanolfl rasource^ are adeqyOTs* 

A progrtiS wists TtT the €valuat 
of the* agefty^- a standards • 

The initfial evaluation ts at the 
CfiOis requestv - , v^^'L^ / 

ExparifhentatiOT and; innovation 
> are encocjtaged* :?'^ v \ 

agenqy pequlpes a jprogtam For. 
the evaluation of outputs # * ^ } 

Ownership and /control art publ Ished. 

Fees are reaaonable* ^ : i; 

Current aceredttstlon statusM and 
dates of next review ti^e published* 

All direetl^JJ affected groups arte ^ 
represented on tfife 'governing board* 



Indbx of 
If^ortance 

- 77.8 
73 
73 



70 



70 

It- 

-in.. 

# 



&6 

n 

62 

46 



• < ■ ■ ■ , fConfdi. , , 

* importance of the •Criteria 

, — Index of 

Critar ion (abbreviated) , ImpbrtaneeS 

* • ■ ' ■ « * 

aSii PrtBecr^ditation is related to 42 

ntcrPd It 3t inn, ^ 



^ bJv A /isiUng team member is present 56 

t^e evaluat ion of the report* 

^''See i,ngti/ page 103* ^ = 

The vaiues in Table 10 ^r^ directly interpretab le * Every 
; eriterion with n value greater than 50 was judged more important 
.than "'or Moderate Importance" by at least half Vm directors "after 
Vt»ie numb*6f of those judgments was reduced by the .number of Judg- 
mfffiit? lower thgHr "Mj'r Low Imporianco." Vs'lues greater than 50 
.tHerp fore 'indicate ' consensus judgment of inportance higher than' 

' ' ' ■ ■ ■ ' . ^ 

"moderole;" Three crUeria fell beTow that value; the others 
.ranged up*#ard frc^n 59, - > ^ 

i . . - \ - . 

The five eriteria most often juc^ed high ih importance: ate 

^ •■ • ■' ' .. .. . . , g 

tal?ited to Evaluation Pfoceduresi StandafdSi and Due Proeess . 
judged smon.q the most important were three criteria directly 
related to Que Proeess — the right .to appeal » the requirement of due 

process berore acGredl.tat Ion may be withdrawn, and the chief 

■■ ■■ • . ■ ■ ' • ' ' 

iHiiiyi^ljOlIiSe » /MM^.^ respond to the^ vis i t Ing team^s repor t , 
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The placement of Due Process lo^t in Importance among the general 
qualities seems to be an error possibly due to a difinition of 
the general quality that lacked clarity or scope, or both. 

Criteria^ asspciated with Standards are widely scattered among 
the criteria of Table 10, One limited criterion, b2iiA5 requiring 
that standards be published, was among the top few in judged 
importance. Criterion b2iil| requiring that interested groups be 
Qiven an opportunity to comment before standards are revised, fell 
slightly above the middle. Criterion b5--for a continuing program 
of evaluation of the agency-s standards— was judgeB somewhat below 
the middle of the 30 criteria listed in Table 10, 

That the greatest number of agency diTectors unequivocally 

ju'dged standards to be. the most importantJ(general quality^ despite 

- I' ' „ . 

the mixed ratings given both the related criteria and the descrip- 

^ ^ . ^ ".V ( ^ \ 

tive- statements assoclattd with them, suggests that some important 

aspects Q-f standards mgy be negleete'd in the recognition process. 

An alternative possibility ib that although the concerns for 

standardsr^may ha^&fe bteen given appropriate attention in the recogni^ 

r . 

tion process, the related statement's were issociated not with 
Standards, but with Dije Process and Fairness, Fairness appeared as 
a general quality in the later, mote complete factor analyses, but 
in the earlier analyses it had b^en scattered mong Integrity, Due 
Procf ss , and St andardSr These uncert^^ 

^.some of the moat important qualities qf agencies, as indicated by 



113 



the judgments or thi^ir impottance that conflicted when. applied 
•. at different levels o f general ity, 'indicate a need for greater 
.Glarity in the general agency qualities which the criteria in 
some sense define. If the kind of attentian an agency gives its 
standards is Judged to be one of the moat important aspects of the 
evaluation of educational quality, the criteria relatBd to atan- 
dards should show an equivalent level of importance. 

Cqnsist_ency of Directors^ Judgments Across Agency. T^-pes 

The diversity of accrediting agencies in size, experience, 
range, and type of,program| evaluated, type of organizatibn, 
and other qual it ie^ , suggests that directors of different types o f 
agencies may differ consistently in what they consider important. 
For example, directors of agencies that accredit medical or allied 
health programs can be expected more often ■ than directors of 
nonmedical agencies to be concerned with clinical education, with- 
training for a clearly defined occupational field, and with licen- 
•sure and certification. The differences that appeared in the 
judgmerius of directors of medical and nonmedical agencies are 
shown in Table 11. 

The Judgments of importance shown in Table 11 are in terms of • 
cumulative percentagBs . ■ The figures indi ate the percentage in 

Of importance plus the percentages in all lower categories. This 
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TABLE 11 

judgments of Importince by Directors 
of Medical and Nonmedical Agencies 



Agency 



Ratings of Importance 
(cumulitive percentages) 





Statement 

• 


Type 


N 


1 


2 


3 


4 


5 


6 


7 


5^6 




1 IBQ 


An 














inn 




with occupational trends. 


Nonmed 


44 


2 


2 


11 


14- 


34 


63 


100 




Most Droarams in it^ fipld 


Med 


38 




3 


3 


g 


21 


37 


100 








HQ 


g 




in 


4fi 

HO 






1 nn 


661* 


The agency is broadly recognized 


Med 


40 




2 


5 


■ 8 . 


15 


25 


100 




by educational institutions ^ 


Nonmed 


47 


2 


2 


a 


21 


36 


62 


100 




licensing agencies and profes- 






















sional Tields, 




















2. 


Ihe agency's activities are ' 


Med_ 


39 






3 


5 


13 


38 


100" 




neither too' Broad nor too 


Nonmed 


49 


4 


8 


10 


33 


49 


71 


lOfl^S 




limited for the need it serves. 




















190. 


Visiting teams include at least 


• Med 


37 


5 


8 


14 


24 


.38 


60 


100 




one person not connected with 


Nonmed 


47 


4 


4 




6 


6 


30 


100 




the agency. 




















572. 


Conti^ ing compliance with 


Med 


40 




£' 




5 


20 ' 


39 


TOO 




standards is mpnitored 


Nonmed 


46 


2 


6 


13 


'l7 


37 


67 


100 




through regular reports. 










/ 










54. 


Institutions or programs are 


Med 


40 












12 


100 



Tegularly reviewed at reasonable 
intervals. ' 

582, Institutions or programs are 
free to use a variety of edu- 
cational methods. 



Nonmed 



Med 
Nonmed 



49 



40% 
46, 



4 10 39 100 



10 ,28 100 
6 22 54 100' 



700, Alii significant , interested 
gtoups are rBpresented on 
the policy-making body. 



Med 37 11 16 36 46 100 

Nonmed. 47 6 11 21 34 53 72 .100 
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-form of presentition permits quick 'comparisons between the overall 
judgments of two groups by examining the largest differtnce within 
any ^category * When two groups, each of about UO persons, are 
compared, any diffirence in the cumulative distributions smaller 
than 25 percentage points is likely to be due to random fluctua- 
tions unless independent supporting evidence suggests otherwise,^ 

Directors of medically oriented accrediting agencies gave 
more importance than did others to the relationship between 
accrediting standards and occupEtional trends (Statement 556) • to 

the agency ' sabil ity to attract most programs in the pertinent 

...» ■ \ ^ ^ . 

field (Statement 6); to its recognition by educational, licensing, 

tand prpfessional bodies CStatement 661)i and to keeping its 
activities within an appropriate scope (Statement 2), They gave 
les& importance than other directors to the presence on visiting 
teams of at least one person not connected with the agency (State- 
ment 19D), These views seem appTopriate for directors of agencies 
closely tied to occupations that have well established licensing 
or certification procedures and training programs with highly 
atructured curricyla. The other statements in Table 11 show 
differences large enough to be suggestiive, but their content is 
not consistent enough to merit futher attention. 



_Th a Ho 1 m gg^^-v=«-Sm-i=roQ V— t^ s t ter— fehe— s Ag nd-fi&anee^ l^d-i~f-F^re nces^ 
between CLCTulative percerttage distributions, based on two samples 
of 40 cases each, shows differences of 27 and 30 percentagt points 
to be required for statistical significance at the .10 and .05 
levels (Walker 4 Lev, 1953, pp. 426-A28), v 



Except for the ninfe statements in Table 11, the directors 
of medical and nonmedical igenciei were quite similar in their 
judgments. 'Issues in the areafe of evaluation proceduresi due 
process 5 and fair practice were all viewed similarly by the two 
groups of directGrs* 

Similar comparisons were made between directors of other types 
of agencieii recognized and unrecognized^ those with less than 
five years of experience and those with more than 20| those con- 
cerned predominantly with pre-baccaliureate or nondegree. programs 
and those that accredit bachelor's and graduate degree programsf 
and private and state approvai agencies^ While a few isolated 
statements showed sizeable differencesi they were too few and, 
too scattered in fneaning for significance* These comparisons 
demonstrated that, regardless of the°'type of agency involved, 
eccrediting agency directors^ with few exceptions, agree about the 
agency characteristici^ that are most important in judging the 
quality of education offered by programs or institutions* 

COMPARATTVE JUDGMENTS FROM -^VARIED PERSPECTIVES . 

Accrediting and state approval agencies serve a variety of 
constituencies. The validity of an evaluation process granting or 
refusing an agency recognition must reflect the Judgments of its 
constituencies* In addition to the direcbors of the ^encieSi 
other important and knowledgeable groups are the. administrators and 
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faculty members of the institutions and. programs evalueted by 
the agenciesi state and federal of ficials , staff members , and 
leg islators^ojicerned with accrediting and state approvalf and 
miscel laneous. 9 there involved with accrediting in a vsriety of 
ways. The judgments of persons in all these groups were considered 
necessary in determining whether the recognition process takes 
account of the most important qualities ^of accrediting agencies. 

Because the requirerf judgments were long .and demanding, an 
honorarium was considered necessary* As a compromii'e between a 
larger sample' and cost considerationst the judgments of about 50 
persons in each of five groups was thought to be an appropriate 
number. That number of Judgffients , i f carefully made by a large 
enough proportion of the people contacted^ was expected to be 
more informative than a greater nunber of responses from a small 
proportion of an initially large group. An 80 pefcent^response 
rate was desired, and as Table 12 shows, was almost reached, 

■» 

/, ' ^ TABLE 12 \ . ^ 

Composition of Expert Groups 

, ^ Number Number Response 

Category Solicited \ Responding Percentage 



Institution and program 



administrators 


62 - 


51 


82.3 


F acul ty niBmbers 


50 


38 


76.0 


State officials 


72 


58 


80.6 


Federal officials 


29 


17 


58.6 


Others 


63 


51 


81,0 


Unclassified 




2 




Total 


276 


~ 217 


78.6 
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Because federal employees could not be piid honoraria from a 
government-funded contract, they were not offered that inducement. 
Their 59 percent response rate cbmpared with an 80 perpent rate for 
the other four groups suggests the effect of an honorarium. 

The persons asked to participate a^^nformed judges about the 
accrediting process were identified through several steps. Nine 
perions connected in various ways with accrediting and state 
approv al--th^ President of the Council on Postsecondary Education, 
the Chairman of the Council of Specialized Accrediting Agencies, 
th'e directors of a regional- and a speclalizefl agency, and others 
who had been active in or were scholars of a'ccreditlng--were asked 
-to list persons knowledgeabli about accrediting in each of the five 
categories listed in Table 12. ■ .Those who appeared on. several 
lists were then sent the names of the persons already listed 
and were asked for additional nominations. Gaps, such as an 
underrepresentat ion of faculty members and administrators from 
proprietary Institutions, were filled through specific rejquests for 
nominations from appropriate agency directors.^ The staff of the 
Postsecondary Education Department of the Education Commission of 



the :States provided most of the names of the state officials and 

some names of federal officials,. Only Z? federal off iclS's, were 

" ■ ■ . ■' . . ■ 

identified, however I who were not connected with DEAE, The list of 

. _ . 2— _■ 1..^ _ - ' ^ 

admlnistriforir was' iafger than could be accdmmbdatedV so de^ 

were made randomly from overrepresented types of anstitutlons and 
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The final group of 276 informed persons was not representative 
of any definable population. It was a heterogenBoua group that 
included perions expected to' have^tat least five differing but 
in formed, views of thi sccrediting and state approval processes. 



Since these judges were not as familiar as the directors with 
the operation of accrediting agenc-ies.'and therefore could not make 
their judgments from -the context of a particular agency, they were 
asked to rank a number of stat'ements rather than make absolute 

■ Judgments.,.ab3ut e^ach one. They were also given brief descriptions 
of^different types of agencies to. which their judgments were td ^ 
apply. This permitted Judgtpents to be made within a reasonably , 
explicit context and to test whether the judged importance of a 

{particular characteristic varied according to the type of agenc.f to 
which it was applied. " ' . 

The questionnaire consisted of eight pages, each with .a one- 
or two-sentence description of a different type of agency, plus 15 
• statements char'acter istic of that agency. The judges were asked to 
identify each agency ' a 'five most important and five least imjportant ' 
characteristics, then to pick the two most important from the 
five already, identified as high in importance and the two least 
^important from those identified as low in importance. Five state- " 
ments remained in the middle. TReSfr-^anks" werf-converted to scores^^^^^^^ 
of 5, 4, 3, 2, and 1 from the most to least important categories 
respectively. ' ' 
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A total of 54 gtatements were organize*d into /groups of 15 
that, were descriptive of each of the eight hypothetical agencies* 
Thirty-three of the 54 statements were repeated for three different 
agencies, which permitted the effect of the type of agency on the 
ranked "importance of the statement to be determined. The mean 

if . * 

\ ■ \ - 

ranks of the 54 statementi are shown in Appendix C* 

• ■ / 
The statements did not differ appreciably ifn comparative rank 

within different agency conte5<ts. Statfemen^' 240, for example, 



requiring' a self-study and on-site review, |receivpd mean ranks 
within the 1-tp-5 scale of 4J2>. 4*2, and 4iO fo'r, respectively. 



an agency in an allied health field cpncernejd primarily with 
bachelor's degree programs, a small sta^a approval agency for 
degree-granting and hospital-based nyrsing .progrms, and a state 
approval agency for vocational progr/ams. The evaluation of 
institutions in terms ^of their own objectives (Statement 332) was ^ 
given mean ranks of 2*9, 3*0, and 2*|7 for the state vocational 
agency mentioned above, a regional/ accrediting agency, and a 
large"^' agency that ^%redited graduate medical programs. Among the 
33 statements ranked in three separate conteKts, only one state- 



ment showed a spretiaB--''Krge^'M ffiur-tenths of a polnti 2*1, 1.7, 




and 1.7* Qudgments of importance tlearly did not vary widely with 

■ ^ ' ■ 

differences in the type of agency to which the statements were 



applied* Carrying out an active program for the improvement of the 
agency's evaluative proceduris, for example, wai ranked near the 
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middle in importance whatever type of agency was being considered. 

ThB collective Judgments of all five groups of informed 
persons were very sijnflar to- those of the agency directors. The 
rank order correlation between the two sets of ranks for the 38 
statements that were common to both questionnaires was A few 

statements werrfftjudged differently by the two groups, and the 
subgroups of informed judges differed among themselves in several 
ways, but the overall similarity in viewpoints was remarkably 



The first six statements in Table 13 were rani jd higher in 
importance by the agency directors than by the other judges. The 
first, Statement S61, showed by far the greatest difference ;in 
ranking between the two groups. Having the agency's authority 
/broadly recognized by appropriate institutions jnd professional 
groups i^s ranked fifth among 38 statements by the agency direc- 
tors, but well down at 27t^^ by the other judges. Further, that 
attribute had beeri fudged more important by directors of medical' 
of nonmedical agehcies^. The agency directors 'alscWere niore 
concerned With evaluative standards and procedures than were the 
faculty members, administrators, state and federal officials, and 
.others. The group of outside Judges, however, gave comparatively 

1 " ' ^ ' ■ ■ 

3WrT-mpTfFffEi^'h^Ffthrigin?^ 

■ - i . ■ . " " 

procedures,, and the adequacy of the agency's staf/, as Is indicated 

by the ranks of the. last six statements in Table 13. As noted 
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earlier, howev,pr, the two groups were quite similar in their 

overall judgments 

TABLE 13 . 

Dirfarences in Ranked ImpDrtance 

Ranked Importance^ 
Other 

Statement Directors Judges 

661, The agency's authority is broadly 

recognized by educational institutions, 
appropriate licensirTg agencies* and 

professional fields* S .2^ 

.591. The agency's decisions reflect fair and 

consistent application of its standards. 1 9 

^52, The agency regularly reviews its standards^ 
policies, and procedures by inviting sug- 
gestions from various interested groups. 11 20 

. 21. The agency s purposes are clearly stated ' ' 

and made publicly available. .1^ 22 , 

382, An active program to improve evaruative 

procedures is carried out, ^ 15 25 

332. Institutions or programs are evaluated ^ 
with respect to their own purposes 

rather than in comparison with others, ' 20 29 

470, The agency reports the specific jreisons 

for adverse decisions to the chief e^cecu- 
\tive officrjr of the institution or program 
and offers an opportunity to appeyl. 12 4 

460, Accreditation or approval is withdrawn ^ 
only after due process^ for cause, and 

with due n'btice; , '13 5 . 



50. The stiff is able to carry out the . ^ 

agency's activities with care and 

without undue delay. ' ^ ^22 , 11 



EKLC 
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* . - . ■ TABLE 13 

' (cont'd) 

Differences in Ranked Imporlahct ^ 

Rankad Import: ann^^ 

' . ^ Other 
Statement OirectDrs Judges 

441. The chief executive officer of the, evalu- 
atejd institution or program is given an 
ppportunity to respond to the visiting 

team report* 21 13 

380* Standards are not revised Until after 

intt^rested groups have had an oppbrtunity 
to react to tht proposed reviiions* ^26 15 

- t * 

490. An institution's or prGgram's iccrtdiL- 
ation statuH cannot be Qhanged nhile 
. an appeal la pending. 34 23 

. _ * ■ ' * ^ ^ . ' 

Figures indicate the ranked importance in each groupi i 1 indi- 

C;ating that that statement was ranked highest in impprtance by 
that group, and a 38 indicating^ that, that statement was ranked 
lowest* ^ 



The judgments of four of the?*flve groups that made up the 
informed judges also were cQmpared. Federal officials were not \^ 
compared wl.th the other cfroups because of their low numbers// 
Again, differences of 25 percentagi points or more in the cumu- 
lative d Utrlbutions are large enough to merit attention. Differ-, 
ences of that maghltude in several relat5?d statements constitute * 
raasonabI> atrong evidence of a notable difference in perspective. 



Two suth groups of rtelated BtaUments are shown in Table 14, 
Where tHs citrtgories of Low, Mlddiep and High show Ihe percentages 
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who ranked each statement anon, the bottom five, the.middle five^ 
or the top, five in import artce. When the top two and bottom two 

■ i .. . - 

rankings ware odued, a n^e-pQint scale was produced, and when the 
samff person ranked a statement in three different contexts, a scale 
with venues of from 3 to 15 was prodwed. Air the stiteniMfcs in 
labie J4 showed differenees bet^en at least two of the four qroiups 
of nl 'iea^t 25 percentsgg points son|g*ere on that J-to-l^ scilf# 



Col lapsing the 13-point scale to the three categories .if figure 1^ 

J- \ ' ■ ^ ' ^ ^ ^ ^^^^ " 

reduced the aizei of the dirferefices between groups* 



fhe first* eight statements In table 14 deal wi^h aspects j^f 
agency evaiuati^^ pTOcedures--the first five with their impiernin* 
tationi and the next thrae with procedures for protect Ing tha 
interests of the eviluated prograi!! or institution* The st^ttmftnts 
within these two groups elicited sirnilar patlerns^ft response* The' 
rirst group of five statements tended to be ranked high mdst often 
by Others and Administratore. The former were critics of accred- 
it ing^scNrl are » Inforfnad persons, staff wpmbsrs oft professional 
assaciat ions * andyother knowledgeable persons not sisociated witHi 
accrediting agencies f col leges i universities^ or goue'rhmental 
agencies. Ihe lattet were ^administrators of corieaes, universi- 
ties, propriietart sehaols, and training programa, rhese five 
statements were ranked high least often by state officials— 
cd^miBaidhers of higher educet^on, staff members of coordlnQtif^ 
c^iifltitteest legAslstors, and mOTbers of licensing agencies* 



TABLE U 



Difrer#nces Across Groups in the Comparative 
.^Importince Aoeorded Agency Characteristics 



Statement Cparaphrasgdl 

129» A proqrsm For traini^ 
'! visiting te^ mmbBtm^ 
is provided. - ^ 

253, The seir-study is not 
;v„ , sufficiently critical 
' - ,or analytic. 



.254* Vislfclnq teams are not 
adequately prepared. 



576* Procedures for observing 
a prpgraifi's avalijptlon ot 
its products are not clear. 



240. A self^study and eJte visit 
^ are required. 



Comparative Importance®^ 



3J2. 



Inst i tut ions or programs^ 
ire evaluated with respect 
to their own purposes* 




441* The chief eici 
T may respOnff^o the v; 
" team report* 



4\0r^"Tfa£,Jlr^^ ft\ft1uaLLori ii 
initlafcea by the chief 
escecutlve officer of the 

. ^instttrtton-^or ^prdgram , "^"^ 



■ ^ ■ -1. ■ ■. 


Low 


Middle 


High 


^ Admin* 


Z3 


35 


41- 


Faculty" 


47 






Slate 


38 


■34 


28 


Others 


31 


12 


57 


Acwnm* 


12 


29 


59 


\ faculty 


13 


45 


42 


State 


21 


41 


38 


Others 


16 


22 


63 


A^in.^V 


12 


29 


. ' ■ * . 
59 


Faculty 


5 


39 


55. 


State 


12 


"10 


38 


Others 


10. 


18" 


72 


Admin* 


16 


37 


47 / 


Faculty 


19 


_J59 


' 22 


State 


2r 


56 


- 21 


Others 


28 


33 


39 


Admin , 


0 




92 


Faculty 


* 5 . 


18 


76 


State 


7 


26 


77 


Others 


' U 


26 


71 










Admin* • 


26 


35 


39 


Faculty 


5B 


18 


24 


Stabb 


60. 


33 


28 


O^heirs 


37 


37 


26 










Admin* 


20 


SI 


29 


Faculty 


29 


66 


5 


^tate 


33 


*? 55 


12 


Others 


31 


53 , 


16 


Admin. 


69 


29 


2 


Faculty ^ 


68 


24 


8 


State 


72 


■ 24 


3 


^ OtheTs 


72 


20 


-8 . 
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^ . TABLE 14 < ' 

' ,y\ ' . (Cpnt'd) ; ' . 

' Oifferancee Aeross Groins in the Comparativt 

Importance Accorded Agency Chiracttristlci 

\; ^ \ ....... ^ ^ ^/ . 

■ ' ■ ' . ■ ^ / ^ \ ■ ' ' ' ^ ; ' . " ' 

V ' ^ ' . V Comparatives Importance' 

Stattmant (paraphrased) Low Middle High 



520. 


Ethical praetlces in 


Admiq, 


51 


41 


8 




rtoruiting and advertising 


Faculty : 


45 




\10 




ar? fostered. 


State 


41 


. 5r^- 






^ ■ ^ ^ ■ , ' 


Others 


39 


41 


20 


541. 


. standards Qf ethical 


Admin. , 


53 


*31 


16 




practice in recruiting 


Faculty 


■ 42 


45 


. 13 




md idvertistng are 


State 


3fi 


52 


, 121 




not clear . 


Others _ 


28 


49 


24 


20. 


Typ.es of prog rami or 




39 


45 ■ 


16 




ins'titutiQns accredited 


Faculty 


J2 


• 42 " 


26 




•■are, clearly defined. 


State 


16 


. 41 


43 




Others 


,35 


37 • 


27 


41 . 


Levels of accfedltation 


Admin. 


24 


55 


22 




are clearly defined. ' 


Faculty 


15 


63 


24 






State 


, 9 


50 


41 






Others 


37 


53 


to 



^Parcentag©! were baaed on the fpllAwing humbets of persons in 
each group 1 . ^ , , , : ; 

■ .^ Administrators^ % ' , % , 

■ V ' . Faculty, menbirs - ^./.; 38v=^ .. . 

^ ■ / State officials ^ ' - 58 ■ 

^-r \ I -■Dthars ' / : "''-5i ^ / ■ . 



In! genaral f the Adminiatrators jnd. Others wtfe more concerned than 
State Of fieials and iomewhat more thaft faculty Memberi,/with the 
detailed procedures "o f the eyaluationrpra 



mmntB related to thr protection of the in^titutldn' a or program's 
intereit$--352, 441 , and 410--Bhowed that Adminlitretora gave ' 
higher importance to that iiaue than any other group. Two Telated 
pain of atatements also ahowed moderite dlfferencta. WitK 
pespact to ethical practices in rtcruiting and advtrtiaing Admin- 
istrators and Others were at opposite extremes in their judgmenta 
of importancer the Othera gave thoae' two stitemehta higher ritlnga 
of importance than did the Adminiatrators^ although moat of the 
jydgmenta o f all four groups were^in the' middle range, Respdnaea 
to the two final statementa showed State officials more iniereated 
than the other three groupa in clarity of function, 

All the statementa in Table 14, and a few more that were not 
-related to any sirtilati statement, sho^d differencei/of at leaat 25 
pe^rQentage points between the two extreme groups in the cumulative 
; percentage diatributipna. Slice four groupa rather than two were 
compared, these differences between extreme groups can be-only 
suggestive, ' . . . 

The aimilaritles among the groups in. the judgments shown in 
Table 14, ind in their judgmenta about the- 24 statements riot 
shownv are more striking Jthan the differences. The differences, 
that reached 25 or 3D or., at times, 35 percentage points between 
groups in the detailed percentage distributions shrank to 10 or 20 
percentage pQinta--Qr to virtually none fo r Stateme nt 41 0-->when the 
scales were collapsed to the three pointa shown in Table 14, 



Though moat of the differences^ still appeared, the range of oercen- 



tsges for any ilngle . statem 
was cpmpif ativfely small. For 
ratings than any othar grouf 
eonQt rned with the prote ction 
But the cons is tenclea in 



ant at any single level of importanoe 
example I Administrators sbowed higher 
on 33?| 441, and 410, .^1 itatements 
of the interests of trie institutions* 
all .four groupi dr judges in their 



/ 



relative rankings of the vprious statements/were far more impres- 
sive than their dlfferen^^., R^ardleis o,r. the group providing the 
judaments, there is no ^ueition that grater importance was attach^ 



ed to the chief exe^^utive officer 'e"^ opportunity tp respond to the! 



VI 



siting team's re^port (441) t^an to the initiation of the firstl 



evaluation only at the chief e^^ecutive, officer's requpst. 



Occasionally, differepces in the variability of responsisrweri 



also consistent aqross groups * For example, a statement abou 
evaluation with respect to the institution's ow\ purposes (332") wa 
mora controversial than one about the opportunity of the chie 
executive officer to respond to the visiting team's report 
The former /Coniistehtly showed more judgments at botti the high anjd 
low levels , of importance than the-latteri which most judges agreed 
belonged in the middle region" of impoTtance* 

Evan though the four groups of knowledgeable observers were 

- ' ' / ■ .■ ^ 

quite different, their, differing viev^ seem less remarkable thar 



their agreatnents. Administrators were more protective of the 



/institution} state of fieiili gave more weight to clarity of 
xfunctdbn or .definition of purppsef md Others, the knowiedgrtble 
persons bbaarvrhg accreditation from outside the agencies and the 
ini.tit^^tions , were more concerned than^ the other groups with the 
evil uat ion proDeas itsei f. But th gee -were comparatively m,inor 
differences in ernphsiis. Overall^ the four groups d;id not differ- 
much in the comparative ranks assigned to the 33 statements that 
-were ranked in three separate conte^^ta As: shown in Table 15, 
: the rank order correlations among the four groups ranged from ,7Q. 
to ,81. ' - > ' / , 

— ' ■ ■ TABLE 15 

" Rank Order Cof relations Amohg' Groups ^ 

%in Judgments of Importance 

>- - " ^ ^ t ■ \ .State ^ . ^ ^ 

^ ^ Faculty Officials ^ Others 

Administratora . ' *81 ,79 *7? 

r Facuny .81 / -78^ 

State officials ^ * ^ ,70 ^ 

Among all four groups of judges ^ the following statements-were 
ranked near the top in importance^ ' . . 

. .55, The agenc,y piAliahis the standarda by which 
institutions or programs are evaluated/ 

40* Procedures for granting, denying, withdrawing 

— -- T - — — re4^^s^ t^ftg ^^c red i t e d^ B tog a t a lTTe 
are clearly described. ^» 



55* The agency publiihei the standards by; whicK , 
institutions or pr^TOTS are evalijated, 

240*^ A lelf^study and dn-siti review ire required./ 
which provide\ for the qualitative asaessmant pf 
an inat itution' s\ or progrini's strengths inrf 
weaknesses. ■.. ' ''■ * 

261. Eafrh progjam o^ institution is give n a -edpy of 
the visiting taam's rap6rt> TwFTifch indicates 
atreng ths a*nd weaknesses jand makes recommenda- 

\l tions for improveffleht'. 



661 r The agency's authority Is broadl^Lecqgniied 
.educational Institutions j ^approprSte lie 
s agencies^ and tha relevarit occupa^fcna.l field. 

128* Written instructions, guidance^ a^Konsul ta- 
tlon are;prpvldedi. for the s^eif-stu^l^ 
: site visit . / \. \ 

" ' . V ■• ■ ^ / \ " ■ ^ ;., " , , ■ . \, ^ . ^- : . > ■ ^ 

Of ; the 80 sta-temenba Judged for importance by thi agehqy directors, 

all but Statement 6'61 of the above. group were in tht top 15 
(Table 6). .StaCement 661 fell about in the mlddle.^^ . - 

Of the 33 statements Judged In three separate eorittxts by. 
the four groups, the characteristics considered the least important 
for accrediting agencies are^n^ted beloWi All were ranked in 
the bottom third^by all the groups. Statement 15| on overlapping 
functions with another agency,, was judged lowest of th& 33 by all 
four groi^a* ^ ^ . . . 



15, The ^agency's functions pverlap with those of at 
least one other agency. 

71. The agency has exterrial audits performed and 
"mlde puBllc* ^ _ ^ 



410. The first evaluation of an irtstitutiori or pro-' 
gram' ie initiated Ih response to an applicatign 
. ^ from the chiif tsceeutive officer of the inatitu- 

283* The^ agency has public repreaentatives on its 
polloy-makir^ bodies. . . 

580; 



i Ex|i^i'iiii6fntatron and inmjvarion are encoufagedV 

;35D. The current approval statuses of institytions 
^ and the dates of the next scheduled review ire 

/ * available^ to( the public* " ... 

" 570, Institutions' or prpgrams are required to submit 

. ^ regular reports indlQating a continuing program^ ^ 

* of self^evaluation* . 

^ Statement 15 was not arnQng' the 80 the agency directors judged, 
but they pliced the bther'siK listed aboye in the lower half, 
^,N,one of the above statements^ however, /can be corial^^^ unimpqr- 
/ tar>t despite their cbmparatlvely low ranking* Because 'the p&^slde 
Judjes ranked the various characteristl6S| they could not avoid 
placing some at the bottom of the list. At/ieast half of the 
agency* directors placed aj;^^ six drthe/above statements in one of 
the top two categories of Imiyortance, " ^ . 



^- . . .' ' . / 

SUMMARY . ' ' \ ^ r / /' 



V, Despite th« generally K igh importance accorded all the state- 
mants, their i^eiatlve J ^lortanc^y'wia clearly and consistently 
.agreed uppn by uir'iuWlly ilJ jgrtups of pereons involved with or 



interested in fiqcrediting* The d'ccaslonal differences in the 
relative r^nka fisslgnted to c^^aricteristlcs of icprt'diting agenclis 
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by groups that, viewid accrediting from diff^^ent perspectivts Wtre 
less impreisive than .the almilafities • This Is sh important 
obierv.itlon . Accredit ing itandards and procedurea have accipted 

■meanings and importance that do nbt vary appreciably, either 
for .rii Kflerent typea of e due a t i onal^,^&g.rjim8-g J. n the views, of 
different constituencies of accrediting The principlea of good 

'accrediting practice that the recognition criteria are intended^ to 
reflect are broatfly jcceptid ^nd can be generally applied. The 
major area of uncertainty \n the preient criteria does not lie in 
the merit of the principlea they embody, but in the translation of 
those accepted principles of good practice into observable agency 
characteristics. Standards, for example, must be clear and applied 
with care and fairness* But igreement is much Ifss on v^iether - 

.stfmdards reqUiire validation igalnst /thicperfidrm'ance of graduates\ 

. _ :\: r ^ - ^ ^ ^ ^ ^ ^" ■ ■ ^ . ^ y 

of accredited jp^ogram^^ f requent-mopitoring by the agency 

through regular reports submitted by accredited programs. The 
specifics rather than ihe geherai principles ar^the source of 
conflicting views and of whatever weaknesses ekist in the criteria. 



\ 



VII. RELIABILITY 



. the reoognition procisa, intended to identify effective 
accrtdXting. and state approval agencies, is reliable to the extent 
^W,. can be carried out with comparativBly little errorT But 
because, it is a complex process involvlrq difficult judgnients of 
about 40 different aspects of agencies and their practices, it is 
vulnerable to error' froiii a number of sources. 



.Initially, the information on which recognition decisions ' 
ireat necegsarily i/tcludes some error. The petitipris submitted by 
agenciep that desire recoghition" are limited antf usually vary in 
the amount of detail they include, yet they must be relied bn for 
much ID f the in formation about the agency. Even such" an apparently J^, 
clearVutvmfltter as the length of time an accreditihg agency has^ 
been operating may hot fae- obvious. The point at which e profis- 
'slon^al. aasoci^on changed its functions to take on an accrediting . \ ' 
role, and w^ethe^ it was an-ffccrediting ^^an^y befdfe^ thr cH 
are-subject to different interpretations. The supplementary | 
correaponden,ce and obst^rvationa by pEAE stqff and consuitants 
clarify uncertain issues, correct some misinformation,' and add new 



information, but all" error cannot be eliminated. 
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In short , ; the basic Information on Which the^ recognitid^n 
aecipion reeta cannot be:a preeisMy accurate or cqmpltte repra- 
aantatlon of any particular agencyX Thi^ source of error, in the 
recognition prdcesa was e^cimlned indl^reetly by aiking agency 
directors ror their Judgments about the Hegree to which ipecific. 



agendy char acter ietics relatea ^o th6 recbgnltlon criteria can 
be abcurataly obsarved-rthatv is, to v^it extents the information ia 
. intrinaiqally . vulnarable to error . A mora dirtect check on this 
iource o^f er.ror--sach aa carrying out a more a^^Xanaive set pf 
ob aar V at iona b f a number of Agencies aimultanebuslyswi th but 
, ifidapendently of DEAE"Was not feasibla* Cdat |_is well as) the 
inabiilty to repeat ■earlier DEAE evaluitions, prevented it. ■ t'i- ; 

-> 1^ . •- ^ 

Another way to identify error is to Compare the hatura of 'tha 
information at dl^ferant points in the racognitlon proceaa. ^Huch 
o^f the information atart^ with ^ C^^ petition foT ^M 

tibn. Thia leada to\a DEAE ataff report based on .tha ^plitition plua / 

^^^^ = . :i. ( . ^ \ ^ ■ : r , ' 

additional informatldn gathered directly by DEAE* Tha fairly 
extensive staff report things .prasanted to the Adv^sdry Committee ; 
and conatitutee, with* further oral diaeuaaidn and any additional 
Informatidn provided by the agency, the pr imar y ^baaia fdr tha ^ 
.Committae^s: racommendation to the Cgmmi^Sioner > Differences 
between. the information in the petitions and the ataff raports 

the staff in v Isits , Interviewa , and . obiervat l^ha * While the 



135 



additional informitioh arid the error ofttncsnnot be easily iepar- 
ated, cbhsistency bitwetn petitions and rtporte li Qne Indicatioh 
that comparatively little error has entered betweerf those t>^d steps 
in the process* \- ^ : 



^ FinGny, error enters the prooeis aa\the^ Advisory Committea 
ibiorba .the Infcrmition from the DEAE staff report , idds infprma- 
tion from Tepresentativep of the agenoy and others appearing before 
the-Committee, and re;pches its deoislon* No group of 15 people 
can cqmpletely avoid the influence of Individual ^eraptct^vis 
or predilectiona in rnak^r^ aollMtive judgments about anything as 
abstract as an agency's Ability, to determine educational quality. 
The careful specificatiQhjWf or quality of the informa- 

tion provided, the gpportu^ity for; representatives of the ^.agencies 
to present their viewSj and the competence and varied baukgroynds^ 
of the Committee members minimi completely eliminate ^ 

error ;in the recognition- decksions. Advisory Committee error will 
appear in inGonsistenciea nrdm ^ear to year in xht nature of the- 
agency charicterlstics that arte most influential in thi recognition 
decisions. . \ 

. . • ^ ^. ■ : 

JUDGED. OBSERVABILITY 

• Sixty of the 80 stateme 
for impbf tance were placed xn 
JudgmentS'of the accessibility 



t^a tJ,.ag.eQc.y..=di£eet^^ 
a second questionnaire to elicit 
of t^^e agency characteristics 



to obserOation.i The 20 Qmlt^o statementa were all cleirly^"-" 
obsarvable , e ,g "The. /agency publishts the standards by which 
inititutlQni or programs mt^ .evaluated*" Howaverj to assure a, 
realistic ipread in the responses,, several afiatementa of obviai^^ ; 
obitrvible characteristics were ''retained in the remaining 60, ^ 



The 20 omittad statements were repiaced by 20. other itatemi 
usually related to the same criteria as the omitt'ed statements/ to 
bring the total^ number to be Judged back to 80. 

The ageney^ directors, judged each at atement on a cfour-^pQlnt 
scale of observability, with the four scale points .defined' in the 
following terms: ^ ^ 

■ t ' • ' ' ^ * " ■ ' . . ■ . i ^ . = ' , ■ 

1* Rare ly Dete rm inable * The statement 1B| in its nature> 

difficult \,to support; the necessary supporting infofmation 

■ cannot usually b^ae^ired in a brief visit to an agency, - 



2. . Raquirgs Astute Judgment . Tha statement is sup'pbrtable 
but remfiree experienced, perceptive observers and careful 
: jijdgment\ equally observers might disagree. ^ v, 



3i Requires Some%Judqmerit * The statement requires the 
. ' eKBrcise of some judgment , but ^most observer^: would come . 
— —t o the s a ffi^-^ 4id gnian t- 8^t# 'F— e-b-r^#-^^ 



agency's records and activities. 



V 4. Readily Apparent. The fiKafcBm#^r>cifif^t.o ip„ 
- of tfie agency that a^e either a matter of publle record 
. • -pr ate resdllx apparent to an observer Famlli 

If tf^ grsup of Judaes is in£prff>ed as IHt directors of eec^a^d- 
/ iting agenciea ware to agree that paEticulir igency character^ 
ware difficult for knowledgf a^e persons to obstry^i the :rtli^^^ 
bility of Qbgervatidns of those Ghi&teflstiDs would be in serious V .. 
doubt, FufthBr, if most 0 r the statemfntf in the staff «POfts 
that referred to a particular erlteri^^ 
.to bt di^eult to observe, the reliability of assessm 
criterion ri^ be questionable; While ^ana^ directors* i 

judgmerits tf^Wseri*^ without errdr , ; they arp legiti^ 

,.matfe indicator^T the probable reliability of some aspects of 

the reeognition process. The questionnaire and its reisults are 

: ■ ' ■ ■ ■ ■ ^ ■ ^ " \ " ' r ; \ ■ . ■ ••■ ■ ■. , ^ :^ • / 

presented in Appendix ' 

■ : ^ '"-^ V ' ^ ■ " ^ ■ . ^ ■/ \ . \, ■ ■ ' ■ ■ : ' ' ■ 

V The several Statements left in th^ questionnaire tha^ de- 
^££i^^£^chirecte assumed to be readilry observable Were- 



Iri fact judged to be so by the agene^ directors* For; eKamplfe, 
8? percent of 93 responding directors agreed that *^The types and 
levels of inatUutions P,r programs accr fed tte 
n-potrt-ishpd^cOT 

i^sticv Ten percent said ^ts observation: required some Judgment. 



^The^ ort;e person who - Said ijt required BBt^^B^u^mm^^m&^- b^e Sp^ 
- fedusing oh tKe phrasti "cleatiy dtfl irf^ile feht others aecipt^ 
^ted the ejcistence of pub^tished docunenfcs as prqv 
evidence.*- 0^ ^ * .^l ■; V 

. - Of the 80 .8t;ateffients ^ 31 )^ere ^udqmd by a half or more -of; the* 
agency directors as being; readily lapparertt-^v ^Anoth^ 
beli^yed by at least, a htl f tff the dlrectcra to require tfnl)^ 
moderate judgment* The remaining 13 etftements referr^ to^f^ihcy 
cr>aracterlstic.s that a halT or more of the directors thoi^ht ^ere 
rr^rely determlnible or required astute Jui^ment* Since 20 addl*^ 
;tion%l atatemerjt^ hid bien excludtd as beir^ o^ 
a AdVal flf 87 out of TOO nonrandw but maderitily^ representative 
©t&tementstfiet appeared in the DEAE staff r^portii re^tred^^ to 
agency attributes that were reasonSttSy cspe^^.to^ 

^'tion pToceas does hot rest, on qualltlas tl||i^i^sre esbterlc or 
^dlTf icult to observe* I that the case,- the reliability 
the proees^ wbuid be In dpAJbt* . ^ / : M : 

. . ■ ■ ' * - , " .- * '' . ' ^ ' %■ -■ ' ' " 

NrvertheiessK 35 of the statements referred to qualities 

judged d^i/fficult to obaerve* and decifeiqna inflwnced^^^^^ 

thos# 13 cJOal itles would have questionable reliability. In Table 

lA/.thos^e statements Vnd the 3T Judged readily accessible to 

Tobservatibn are pressw.ad^W^ t^eir j.udjcjl^ 

bbservabllity * ,The median values .©tate the point on t hi*' scale 
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"J^/Tt' t to U below wnich half the judgment "re 1 1 - -that is, the 
midpoint of the judqmentii . 



rritef'i 




b5|i 



bin 



b2i 
b3v 

bJvi 



lABlE 16 

Cb^^rvabil ily of Agency Attributer^ 
Statement 

^^ndily obaervablg ■ ; 

^-^K ihe chief ^^etuiive officer of^hc 
evaluated instiLution or proQfam is 
qiven an □pportunity to respond to 
the visiting team report. 



20, The aqfency's purposes and/or statutorv 
authorUy art cltarly stated and made^ 
publicly Bvm Uble, 

21, ma types and levels of jHstitutians ' 
Of programs accredited era elaarly 
defined in published dMuments, 



Median Judcjmcnt 
ih ^ 93) 



I 



261. Each instiLution or program is given 
a copy or the visltinq team's reoort . 
which indicates strengths, weaknesses, 
and recommendations fop improvement, 

283. The agency has publjc repT-esentBtivefs 
on Its policy-making bodies, 

aJO. At least one member of the visiting 
* team is required ■ o be present at tpe 
evaiuBtlon of the visiting team report 

iOO, Adverse decisions are roilowed by a 
hearinq at Nhich .representatives of 
the arfected institution or progi 
may Bppear. 



3,94 



3.92 



3.89 



3.88 



3.86 
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TABLE 16 
(cont'd) 

Observability of Agency Attributes 



Criterion Statement Median Judgment 

" (N - 93) 

Readily observable ' 

b3ii 272, Visitinq teams enqage in discussion 
with faculty, administrators, and 
students. . 3.82 

a2ivA 124, Institutions or programs to be 

evaluatsd may recommend or reject 

particular visiting team members, 3*81 

b6 370. Institutions or programs are required 

to submit regular reports indicating a 
. continuing program of self-evaluation. 3,79 

^ a3i 40* PrQCedures for granting, denyihg , with- \ 

drawing, or reinstating accredited status / 
at all levels are clearly described, - ^3*78 

a3i 41, The definitians of each level of accred- 

itation or spproval granted are clear 
and v^ll-difrerentiated * 3,78 

b1 5, The agency is the only accrediting or 

approval body for its purpose in its 
reg ion . 3*77 

bJvi 460. Accrsditat ion is withdraw only after 
due process? for cause,, and with due 
notice* ' 3*76 . 

b2iii 380* Standards are not revised until after 

interested groups have had an opportunity 

to react to the proposed revisions- 3,72 

b3ii 281. Sblicitation of the views of students 

is part of the accreditation or approval 
process * 3* Tl 
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TABLE 16 
(cont'd) 

* Observability of Agency Attributes 



Criterion Statement Median Judgment 

(N ^ 93) 

Readily observable 

d2 743. Members of the decision-making body 

do not participate in discussions or 
decisions affecting^ institutions with 
which they are affiliated* 3,70 

d2 484. Appeal procedures include sareguards 

against biasr e . , no one involved 
in the original decision may serve 
on the appeal committee. ^ 3.70 

b2iv ^390. Written procedures provide for fair 

and timely treatment by the agency of 

complaints against institutions or 

programs. - 3.69 

a3iii 240. A sel f-study and on-site review are 

required that provide for the qualita- ^ 
tlve asiesiment of^an institution's or 
program's strength^ and weaknesses. 3.69 

a3iii 260, Visiting teams ar^^concerned with 

instruction, resojurcesi managmbntj 
and student servi|es| all four areas 
\ art covered, s^/ ^ . 3,66 

b3ii 270, Faculty members are given opportunities 

to make their views known to the agency, 3.65 

b6 572. The agency monitors continuing compliance 

. with standards through regular reports 

submitted by institutions or\programs. 3,65 

b1i 282. Members of appropriate academic, occupa- 
tional, or professional fields.^have a 
role in the accreditation or approval 
process. ' " ^ 3.64 



\ 
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~ TABLE 16 

(cont'd) 

Obaervabiility of Agency Attributes 



Criterion Statement . Medim Judgment 

(N = 93) 

Readily observable 

b5 552* The igency regularly reviev^ its 

standards J policies, and procedures , : 

by inviting suggestions from various 
intefeatsd groups* 3,61 

a1i .6. Most pr^rams or institutions in its 

field or jurisdiction seek accreditation 

or apprroval from the ag'ency, \ 3,58 

b5 642, Information regarding standards and 

guidelines for the conduct of site 
visits is exchanged with other 
accrediting agencita* ^ ■ . 3,57 

a2iv 110, The agency selects its personnel in ^ 
. accordance with 'nondiscriminatory 
practices, 3.54 ^ 

a3ii 220. Procedures' for the move from pre- - 
accreditation to full accreditation are 
provided and appropriately related, 3,52 

b9 598, When circumstances warrant' it , institu- 

u tions or. programs are reevaluated at 

intervali shorter than the normal cycle, 3,52 

c1 661, The agency's authority is broadly / 

recognized by educational institutions, 
appropriate licensing agencies j and V ' 

professionil or occupational fields, ^ 3*52 

* ■ ■ ■ . ■ 

Difficult to observe 

d2 712, Through its structure and procedures * 

the agency eliminates politically- 
based decisions'' that may adversely 
affect the quality of education, 2,45 
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TABLE 16 
(cont'd) , ■ 

Observability of Agency Attributes 



Criterion v \ Statement' Median Judgment 

' (H m 93) 



b8 



vpifficult to observe . . 

603* The agency tempers its evaluative role 
so as not to endanger the reputation or 
the accreditation or approval stitus of 
its institutions or programs. , 2,41 



b1 The agency exists solely for purpot^, 

of determining eligibility for federal 
. fundingrit serves no other need. , 2.35 



ERIC 



d2 — * The agency's public representatives have 

vested . interests in the accrediting or 
approval body or its constituents . ^ 2.30 

bli 284. Accfeditat^n or approvir by the agency 

assures thll graduates ape occupationally 
• or profeisionally qualified for appro- 
priate employmant. 2.24 

b6 575. The agency's r#view*bf Institutions* or 

programs' annual^ reports is not thorough 
enough to ensurt that schools or programs ' 
adhere to standards. ^2.12 

b5 334.= The agency's evaluative progrm of its 

standards is .weak, poorly .conceived, or f"^ 
= marginally effective, 2.11 

b4 538. The agency fails to monitor the fraudu- ' 

lent use of student aid funds ,by- 
institutions ^or program's* 2,11 

b3iii 251. The reliability of the visiting team 

report is queitionabl^* 2.00 

bB . Accreditation or approval by the agency 

is not a reliable Indicator of either 
institutional Integrity or viability. 1.97 



, fABLE 16 

(contjd) 

Observability of Agency Attributes 

Criterion, , Statement ' Median ,3ydgment 

(N ^ 93) 

DiffiQuIt to observe ^ . ' 

b5 335« Some of the agency's stanlaards are 

^^questionable or hot demonstrated to be 
valid or reliable, 1,96 

b8 The agenoy'i iGeredltatlon or apprdval 

assures the eKpenditure of student and . 
government moneys in an edycationally 
effective and economically prodgctive 
manner. ' 1.92 

d2 728, The accreditation or approval^conf^rred 

by an agency has been distorted by 
pol itical^e5^Re<jiency* V 1,84 



The Observable and Dif ficult-to- Observe Qualities 

-" - " r~ I, ,s ^ -V - ^ 

The readily Dbserved characteristics of agencies concerned 
matter^ of procedure or structure that are described in public 
documents. The charac.teristicsi. judged difficult to observe 
required evaluative judgments about the -effectiveness of the 
procedures. The foliowing;two statements and their median ratings 
illustrate the difference In^observability between me eKistence of 
formal procedures and their effectiveness. ' - ^' 

572. The agency monitors continuing compliance 
with stand^ardg throOgh regular reports 
. ' submitted by institutions or programs. 

, - ^- ' 
575,. The agency ' s , rev iew pf initltutions ' or. 
programs' annual reports ib not thorough 
enough to ensure that achooli or prograns 
adhere to standardt. 



ERIC 



158 



i ■ ■' ' 

Jhe- first statement,- which refers to a question of procedure, 
hab a median rating of □bservabil ity of 3,65^ which placti ^he . 
midpoint of the judgments- a little higher than midway between 
"Requires Some Judqment" and "Readily ApparBnt." The second 
statement, calling for a Judgment of Kfrectiveness , had a median 
rating of 2.12, barely above "RuquirBa Astute Judgment. " 

' \ ' . / ■ 

For the rr^ost part, the. attributes difficult to observe were 
not associated with any part Icular set of criteria. -Most of the 
criteria refer to an agency's structure and its .formal procedures, 
and the apecific observations assocrated with each criterion tend 
to span a fairly .broad ran^e Qrobseryability* The two statements 
above are il lust rati ve | both ar.e associated with Criterion b6, 
which refers to ongoing; programs of institutional sa^^evaluation. 

An exception was the set of criteria concerr id with the_ 
aqency's integrity. Of sixteen statements in that group (Ap- 
pendix F, p.7), only three were judged resso^ ay accessible to 
observatipn. These referred to the disfluaaification of board 
members in dTCisions Effecting their institutions (Statement 743), 
broad recognition of the agency's authority (661)^ and policies 
protect.ing ggiinst cdnflict of Interest (64). The other 13 were 
all in the lower, half of the 80 statements with respect to observ- 
ability. The observable attributes are procedural, as in thy 
poiicy preventing members of^ the decision-making body from partici- 
pating in decisions about their own institutions* MoBt of the . 
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qualities related to integrity, however, call for Judgmente-- 
whethtr decisions are fair and unbiaaea, whether ethical practices 
ire fostered, and whether members of the decision-making body are 
able to make reliable decisions. 

Overall, the agency directors V yudgments indicated broad 
agreement that most of the criteria, arid the specific character-^ 
istics associated with them, were reasonably accessible to observa- 
tion* The exceptions were _the criteria related to integritys and 
those cbncerned with standards, although to a lesser' degree , as 
is demonstrated in the statements listed in Table 16. 

Dbserva fei 1 i ty and Import an , ^ . 

The .^attributBs judged jnost important to an agerlcy's ability 
to evaluate educational programs were described in Chapter VI* 
They are associated dominantly with procedural issues— in evalua- 
ting prngrams, in reaching decisionsi and in assuring due process 
and fatr treatment. The existence of established procedures is 
both important and observable* At least as important j but much 
more difficult to observe, are the deiired consequences of the 
proetdur es--that only qua! i f led programs or institutions ire 
accredited , that decisions arje fairy and that staff and financial, 
resources are adequate for cirrying out an effective program* 
These general obeery otions "are presented in more detail in 
Table 17, which contrasts the observable and dif ficult*tp-observe 
statements judged high in importance* 
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TABLE 17 



'Observabilily of Important Attributes 



Crittrlon^ 



Statement 



Observeble 

— \ 

a3iii 240, A self-study and on-site review. 

ore required that provide for 
the qualitative assessment of 
an institution-s or program's 
strengths and weaknesses. 

b3ii 272, Visit ing_ teams engage in 
discussions with faculty, 
administrators, and students* , 

b3iii 261 . -^ach institution or program is 
given a copy of the yisiting^ 
team's report, which indicateB / 
strengthsi weaknesies, and 
recommendations for improvement. 

Wvi 46d. Accreditation^ is .withdrawn only 
after due process, for cause, > 
and with due notict, 

a3i 40. Procedures for granting, denying, 
withdrawing , or reinstating 
accredited status at all Jevels 
are clearly described. 

bJiv 44U The chief executive officer of " 
the evaluated institution or 
program is given an opportunity 
to respond to the visiting. 
/ team report. ^ 



Rank Order of 
Importance 
(N ^ 80) 



Rank Order of 
Observability 
^ (N 80) 



1 4 



20 



4 



14 



11 



19 
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TABLE 17 
"(cont'd) 



4 



Observability of Important Attributes 



Criterion 



Statement 



Rank Order of Rank Order of 
Importance ' Observabil ity- 
(N ^ 80) (N 80) 



Difficult to observe 

b8 593, Written procedures oleLrly assure 
that only qualified inetitutions 
; or prograns are accredited or 

■ ' approved. 

b8 591, The agency's decisions reflect 
I -i ■ f^Lt end consistent application i 

of its standards* . , 

bS 590. The agency's published procedures 
are fairly app^lied* its actions \ 
are consistent with^ts procedurea 

kZi 50. The staff ia able to carry out the 
I ^ency's activities with care a^nd 

; withiout undue delay. 

j , ^ ' " /■ ' ^ 

a2ii'/. ^ 72. The agency haa,^the financial 

/\ ' resources to carfy out its 

i , activities effectively. 



11 



16 



17 



41 



55 



m 



43 



44 



The^first Bi% statemeAts in Taffj^e 17, which describe impor- 
bant t readily obseryad attributes of acc^reditirig ag^encies, are 
related to procedurea / and due process , as is indicated by the^ 
criteria ^ith which they are associated as well as by their con- 
tent. All si5c were ranked in the top quarter in ..both importance 
snd observabii ity * The last five statements , which d«al witf> 
integrity^ fairness^ and organiiational ef fectiveness , were iiso in' 
the top quarter with respect to importance, but ranked in the io^er 



half in obstryibility i 



1 
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. ^ In short, then, the collective ^ judgment of fhe agency direc- 
tors and the five other groups of knowledgeable persons ahowed 
a gap bstween the observabil it y o f the e^^ietencs of standard 
operiting or evaluatiqnal procedures and the consequencus of those 
prpcedures* ^This is a wholly understandable di^^tinction. Written 
policies can prescribe activities for evaluatian and decision 
making that are ordinarily sound &fid effective, -md the e^iistence 
of those policies is not hard Ho\determine^ But whether the 

policies, as implemented, do 'in fact result in sound decisions 

i 

is more difficult to assess. The reliability with which |effec;ts 

are determined can be expected to be, appreciably lower tl-aiv the 

^^n^^^^^^y ^^^-^ which the existence of appropriate polibies is 
determined.. - 

^ J'- ^ ^ " ' . , i 

■ r ■ . ' 

. i 

Host- of the criteria cover □bservable, pRQcedural character- 
istics that are widely judged to be desirable^. Some, however, 
address more .subtle issues that^re not readily observabie. The 
most important of the di f ficul t-to-obsepve criteria were a2i and 
all=i, which tjeal, respectively with the administrative effectiveness 
and financial strength of the agencyj and Criterion bB, a oompre- 
hensive one that requires accuracy of judgment, fairhes-s, and 
impartial it'y in the accrediting process. Criteria d1 and ^2„ which 



refer- to autonomy and thlx^prevention of conflict of interest, were 
considered plmosb as important as , those listed in Table 17, and 
they, were also judged, difficuj^ to observe,. 



These Tive drite ria ^ then--a2i, aZii^ bS, d1 , and d2"were 
those where importance and difficulty of observation combined to 
suqqest a need for stronger evaluative or observational procedures. 
Elaboration,. and clarificatfon of those criteria miglit lead to mbre 
reliable observations, 

\ , - 

CONSISTENCY BErWEEN PETiriONS AND DEAE STAFF REPORTS ■ . ■ 

■ \ ■ . ■ ■ " : ■ 

T'wo major stsps in the recognition process are the agency^s 
petition ^ for recognition^ which' statee the .agency - s characteristics 
pertineht to each criterionj and the DEAE staff report to the 
Advisory Committeei v^ich summarizes information about the agency 
again in relation to each criterion* Differences between the two 
documents imply that one is in error* Whether the error is 
in the staff report or In the petition cannot be detarmined simp 
f r om the ex is tence 
indicatr' high rates 



of a difference f but frequent differenceji will 
pf error at some point* Infrequent differ- 



ences will suggest few errors and high reliability* 

The most dirocb way to compafe the iccuracy with which infor- 
matidn in t^^e petitions is transfe^^ed to the OE^ staff reports 
is vto list the consent* of both,,documents and compare them^ While 
not an entirely sa|tisfactdry procesSj that was done^ for a sample 
of 34 agencies evaluated after thi 197A criteria had been 
effect* ^ne diffi^culty was dye simply to the bulk of the petitioh 
documents* In ff/sny cases they were sevtril Inches thick and 
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included- handbooks, annual reports, pol icy" statemenra, and corre- 
spondence in addition to a lengthy narrative deacribingCthe 
agency's way of satisfyAg each criterion. An ETS staff member 
other than the one who coded the corresponding staff report read 
each set of documents and coded them in terms of the same, descrip- 
tive statements used to code the staff reports. Thus two sets of 
.statements descriptive of thg same agency were produced, one based 
on the petition and the. aupp.-.^ing document that acdompanied it, 
and one basHd on the, staff report, . ^ 

Kappa was computed as a moagure of similarity among all 
pairs of agency petiUons and staff reports (see p. 72), In this 
context, kappa-,^ a measure of the degree to which two sfts of 
statements approach perfect agreement from a base equivalent to 
random agreement. Its. range, for all pairs of petitioris and 
reports, was'frdm small negative vali«s, indicating- l*ss ^reement 
in actuality, than would be expected .from random assignment of 
statements to pet-itions .and staff reports, to approxirnately ,80, 
with a raean of ,37'. This; d istribut ion of kappas was closely 
apDroxi|ated whe'ther the simi lar^ty was between stafr reports, 
bet we sn agency petitions, or b^ween staff reports and petitions of 
different agencies. If the descriptive statements were assigmnl 
randomly to the agencies, the''mean yjlue of ksppaV or the mpan 
similarity between agencies,, ,^ould >be close to zero. Thg valije 
of .34.=l3eLwein different agencies reflects the lack of independente 



among the statements. The appea^anct oF some statements as 
dtseriptive of aB^^^ncy may increase or decrease the probability 
that other statements i^ill also apply to that ^ency, for exarnple* 
a Btatement that ah agency fails to comply wH^h a pjrtieular 
crittrion increases the probability that other unf^-^vorabie stafcc- 
ments related to that crUerion will appear and decreases the 
probsbiUtv that favorabl© Veiated statements will appear. Some 
general degree of similarity* represented by the mean kappa ar\37, 
was to De expected* 



The values of kappa between petitions and staff reports of 
^he same ageneies shoulcj be appreciably hu^r, than the values 
bc^tteen petiticnp and si ,rr rW^m^^^^^f different' age^||^s. Ihe 
distr\ib<it ?Q ; of those values**the M valyes of kappa for kiieh the 
pte^.Uioh^ad staff report reFerred to a ecr^ion agandy**ranqi| from 
,15' to ysfi **ith a meao of .47 a^d a ttandard deviation of *1C 
A lthouq'^'^aa standard e^c i^ts agaipst ^hich such values can be 



evaluated, two descriptions t^e aamt object ought to be SLtostarN 
tially more similar than t«o descriptians of diffefent objects* 
The inrr^ase to .67 trcxn an overal l 'mean of ,37^«ebout t^-thirds 
of a standard/deviatson^--seemi less substantial than might be 
pee ted'- . ^ ^ 

fne .timited detires of co^pfiraHn ity hf^twee^^ p^t ttionfi and 
staff reports can be attributed to saveral obvious sources of 
possible error* First, the materiai in the petitions was oft^n 



. . •.c.n;uv«s rtiseursive, and the , result of Idiosyncratic inttrA 
,. : >\ — Ihi' i wutiynitiun criteria. Two persons reaaing 

• . ,;^..,,.n ino sunmnrizinq the agency -haracteristics pertinent 

" -rneria wnuld be ftHpee ted to produce. acmeirfiat, different 
rifi'inns. Supplied with sOTie inrormation that <«f ambiguous 
^ ' ot^fr information that was only marginrt ly pertinefit< to a 
:jr cr iterion (or opffletlmes irrelBvant even though descrip- 
' ..f> f ' ':of-<> fSesiroblii attribute of the agencyy, they would almost 
■' r •■Tpend *^ith different Bummary descriptions. Second, 
-^rH fi-pcrl inc^jded material derived -irbffl. sources' other than 
' V . i.iLurv. 3ucn as a visit to the agerjcy or the observation 
• .I ' liif :; tersff or comm iss ion meeting third , the prdcess of 
petUjunn and the^ staff ;rGports , and assigning 
■ .! i^i or a graup, of ■standard ff' ttements to - th& docyment , wsa 

^.,f^n? .r-r rM^ thiffl ^Durce ^err or W^?^ nnU^vi^f'd by ' 
^ - Qrouft of staFf reports codtd by sevi-ra^ personi. 

' ^ ^ r.4;cMhued bf*twen pairs pf soders assjgnihg. descri5>tive 
' ^ - F p^^^s to the iiame i^tfifr raporr ranged from ,55 ^^t^ with 
^ - ' rif a^^l ^ land arid deviation of 06, V'^^^^ 

- -.^ l nf aiire^N>nt r«aeh#d^rter the eiders had rasd several 
r' , r^^; cn!i-d s??v?^r3l morel disegssed and revised the cat&idg of 
r;.. i^niUfmt^nts j c&dftd several mpre rteportsi reyiewd the 
- oitxer cou&rgi and finaliy ^greW QnanteFpretrticfn?^ the 

* i t; r;f mh? s Vit^^ment^^and procedyres for resold »!»bi4iiitles* ■ 



intrrcader igreement on the_ order oT .80 is desirablt md 

aonrrttimas aehieved,; ^j|t that level is usyally limited to situations 

where ona object to be assignad to one clatsif icition , in a group 

of five or sii^, as when psychiatric patierits are classified into 

diagnostic categories. In the present study, the coders assigned 

30 to 100 statem^ Its to'jich agency from-^ total list of 559, The 

oppoftunity f or d iitgreemenh to enter into, such a situation is 

.obviously great. The utility of judgments that show intercoder 

. ^ ■ . . ^ \ ' W ^ ^ 

agreements r/f the magnitude achieved is therefore difficult to 

assess apart from the results produced by those judgments j in other 
anal vf^s^ where low reliability would be a limitir^ factor. Tne 

. " ■ " i 

succoss of the multidimensional seal ing analysis in discriminating 
amonq ^eneies according to ^^ether th^ty had been recooniied nr not 
indicatas a level of rtliabllity in the assignment 'of attributes to 

aqencifes great enoi^h to produce interpretable results*/ 

■ - ■ ■ ^ / ■ 



Lof\4 



The Ir-yi^l of agreement between the descrlptlom of agencies 
based qn petitions and t^ose based, on staff repdrts/ can be evalu- 
ated in relation *;o the .values of 4<appa associariod with inter- 
coder agreement. Wilh a mean intercoder kappa oy *65 and ttandird 
dev^iitiqn^f .Qftr a rough uppev limit of about .Ip is the best that 
can.be expected iff campating staff reporis apd petitions^ Sme 
value between ,35 and * 70 should therefore be escpected. The 
athieved mean value of ,47 i& stilj not great. The additional gap 
up I© ,7Q repreaenti bq|-h the addiCiona) ir ■ rmation that f^ntered 
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the staff reports and the difficulty in apply ing /stitements 
derived from staff reports to th© more diffuse content of the 
petitions. The obsewed level of agreefnent, however^ is not high 
enough to resolve all doubts about the accuracy of the informition 
oh M'uch recognition is based, 

, A -further study of the source of the dlscrepanciis between 
petitions and staff reports^-whether from poorly prepared peti- 
tions^ from n&^i information entering the process from sources 
other than the petitionSs or. from coding iniccuracies by ETS 
staff—would be desirable. Discripancies attributable to the first 
source would suggest the need for chaises, in the guidelines for the 
preparation of petitions. (In anticipation of this result, DCAE 
commissioned the preparation of new guidiliness which constitute 
Part III of this repoft). Discrepancies that arise from the second 
source may also be reduced by improvements in the organiiation or 
content of the pet it ions , -.^ Cod ing ^inaccuracies are not errors in 
the recognition process, but estimates of their magnitude will 
suggest how m,uch additional error pemains between petitions and 
staff reports. - 

Some errors are inherent in the translation of petitions and 
rtpprts into sets of standard statements. The need to represent 
two slightly diffBrent pieMs of information with the same state- 
ment, and the treatment of two statements dealing with related 
content as totally distinct* are two inherent sources of error. 



■ ^ ^ / ^ 156 ' ' ' ' ' .. 

Comparisons betwten petitions and reports at a level of generality 
broader than the stitements themstlvea would. reduce the effects of 
tho^e two sources of inaccuracy. 

Mul tidimensional scaling. ( MDS ) analyses were there fore 
' ■ % ^ 

carried out wit^h similarities among agencies based on the petitions 
and independently for the same agencies with similarities based on 
the staff reports* The two lources of information are functionally 
equivalent if .they lead to similaT. distinctions among the agencies 
— that is, if they place the agencies in similar relative positions 
in a mul t id Imenjional space* A simple comparison of the two MDS 
solutions, however, based on correlations between project ions .of 
the 34 aqencies on each of the three dimensions of the-two solu- 
tions, would not be appropfriate. Differences in the orientation of 
the dimensions in the ti^o solutions would affect the projections 
and make direct comparisons meaningless* ^ procedure was used, 
therefore J that places the two solutions in a common multidimen- 
sional space and rotates the dimensions of that space to bring the 
two rep/esent^t ions of each agency, as 'close-^ together as possible 
(Kaiser^ Hunka, 4 Bianchinli 1971 )* "Correlations" batmen the two 
sets of dimensions then indicate the overall agreement between the 
two solutions and, in the present study, the degree of congruence 
in the discriminations made from the petitions and from the staf^ 
reports. "Correlations" is placed in .quotation marks because their 
calculation is not based on separate, observations of each dimension 
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for each of the 34 agencies, which would be the usual basis for 
; correlation coefficients, Ytt because they are the coiinei of the 

,||l|||les between the three pairs of dimensions, which is the geo- 

• - ■■■ ' " ■ ■ ■ . ■; '. . . ■ 

■ metric ■interpretatioh of a correlation coefficient, they can bi 

■ ** ■ . . '■ / ■'■ 

interpreted ag such. , " " ,. ' 



TABtE 18 - 



1 



Caprelations Betv«en l-iDS Dimensions 
of Petitions and Stiff Reports 



Dimensions of Dimensi.ons of' 

petitions staff reports 

■ : , .1 . 2 ' ' 3 

1 .78 % .57 ,26 

2 _ ^ ; ^,36 ,75\ ..56 
'J ■ -.51 .35 .79 



X 



All three dimensions ware reasonably cong ruent , -wi th the 
three, correlation coefficients all at .75 or higher, as shown in 
Table 18. Thg general Issues or perspectives that discriminated 
among the aqennies as described'in their petitions were therefore 
clpaeiy related to the discMmirtating concepts in Ihe staff 
Reports;. " ' . ■• 

' The congruence between the petitions and,' staff reports with 
. ■ ■ 't_ ■ • ', ■ 

respect to the general, diatinctions they mike anong agehcies U 
. ,' ■ ■' .. ■ ■ ' _ ' - 

as. high as the reliability of..thB coding process. That, could not 



occur unless the real differences in content betweth petitibni and 
/ ^\ -. - . " ' ■ • ' 

stiff reports, as well as the dlfftrtnces due to coding trrors, 

involved specific details rather than general agency character- 

istics. The important, overriding distinct iorta among agencies 

art therefore reasonably nell preiervtd in the passage through the 

recQgnltibn process* 

CONSISTENCY IN DECISIONS FROM YEAR Ta/YEAR^^ V 

A direct approach to assessing the reliability of a measurt^ 
ftientvprdcess is to exafoine the consistency pf the results as the 
process is applied repeatedly to the same object or to objects 
known to be similar* In th^ evaluation of gsccreditln^ agencies, 
such a process would require that a sample of agencies be ^^valutted. 
independently by different per soni iti about. the same tim? and the 
results compared* The level of agreement between those independent 
appHcat ions would indicate the reiiabillty oT otofieBU. Ihe 
Infeasibility of that direct assessment qf consistency led to the 

use of an approve inrat ion to it, , ^ 

■ ^ ' \. ^ ' , " ' / ^ " .... ■ v \ ' ■ 

% vjha procedure fol lowed was tc^ ^ cr t^^i s.t&n^y rpm year: 

to yt&r in the general qualities that distlnguii^^*^^ ^encxea 
evaluated differently by the rec^nition process, fhe decisions^ - 
th^ aasignmant -of an agency to one of flvt^ reco^nit ion levela , 

■ ■ \ : ' ' ' - r . ■ ' J ^ " ' - -. ^ - ' ^ j- . ' ■ / V . ■. 

from denial to ^^na^nitibn for four years--diu njt retain anou^rt 
infdrmation in themselves to be a use'^ul indicatQt of eohsrttency 



or absence of error in tht, dtciaion pr^ocess. The same decision 
couldlresult from too many differ^int combinations of ^ysncy 
charjcter ist ICS . The itatementg presented the sams problem' 
in reverse, Differ^ent statements referring to similai? agency 
characteriatios an^ produeing..similar results would yive a false 
Indication of lack of congistency* Consistency we^^ therefore 
examined in the T^ays the statements were organized into general 
quaLities, ' ^ 

I he st atements that appeared ^frequently but not ynlv^rsan 
.;in the.DEAE staff reports descr4be¥ the qualities ^f eGcreditinQ 
. agencies on which recognition decisions primarily e^^a ba^ed. At 
times, an unusual deficiency was severe enough 'tp\enter h 
intq^ decision, but by definition that was a r ^^e c^jwurrience* 
The r#cogn i t ion pjcocess In general hanq& on the, cofTS^ ^^\v<^^Uly; 
: recurring cphtent :Gf tKe DEAE' staff repor^ts, which a^e organized 
W^^und th^ published oriterifa> ^ ' ; - / \ ; / 

Par t icular statements" that appeared ^n rVo reports often 

rs'ferred i© different aspects of an agency i^lated to some more- 

..^ - -V • , . -v ■ . ■ " , : , - ■ . - / - 

iqen^tai quality/ - Two suc^ statements for example , that >^fere 

' - ■' ■/ ' - . ^ ' . ^ \ - ' / . / ■ ^ . •■ ■ ^ 

felated to a Ci^mm genersl quality ^ a^ 

Adverse decisions are followed b/ ^ hearing at whiiSh 
^ representatives of the aRfpcted institntifeion or pr^ram ^ ^ 

■ Viilting tvvams include at le^ft one person who la 

\ V not a membf^r of thf agency's policy-making b or ' 
: admihistratzve staff, r 



These statemtnts are clearly dietincti neither implies tht other* 
Yet these two attd seyeTil othera, all r alited to pf ocedurei \for 
assuring due prqctis |/^ere rout^ually rtlated,^^ 1 a general 

sgtney quality* If the recpgnitibn pi^oceas operates rellibly^t the 
stflittmenta that dlacriiTiihata among sgenciea will coniistently show 
similar relationships, indicating that the same general qualities 
.are rtflfcted in the decisions* 

' During the eleven-year .period that was studied ^ two sets of 
criteria were appliedi, .A revised set of criteria was' put. into 
effect in mid-1974, creating a syatimatic^ difference in the recog'- 
nititfn proqesa as applied .befor# and after that' time. Further, the 
composit io^i of thf Adviiory Co chaciged by 25 percent each 

year^ which resulted in a complete t^r hover ^in membership every 
^four years. Slight differences in the nature of the decisionSi the 
effect of differences in emRhases and interpretgti^s of different 
Committae memberai were therefore to be escpected from, year Uo^ year, 
possibly becominq greatei over time as the dominant issues in 
postseconda,cy education changed* Both these sources of posaijble 
inconsistency In the dec islori procesar*the ghiarp chaihge with the\ 
Introduction of the 1974 revisions in the priteria Phd, the gradual 
chahges^^resul ting from changes in personnei ^nd;eircumstances-*-i/pr 
baiancer yin evaluating consistency over time by comparing the 
general qualities oboerved in odd-^numbered years with those 
observed in even*numbered years* ? 



The prbcadura. deveioped by kaiser, Hunka, ^and Bianchini 
(1971) p describfd in the precedlrtg-aec tion, wa^ used fcd com^^ 
- generalv,qualibias across odd-numtiered and even-numbered yiirst as 
expEts.sed in the factor patterns of the dssi^riVtive statefrttnts 
that appeartd. in the 234 staff reports associated with the" 234 
recognition dec ii ions made from 1968 tp 1978. The thrti' 1968 
decisions^ which preceded the 1969 revision of the criteria, and 16 
decisions for which ftwef than 20 .descriptivB. statements appeare'd 
in thp staf'r reports were excluded.- The two iets of odd-ye'ir arid 
even-year decialdns therefore had slightly more than 'lOQ decis-ioris 
in .each. Because a ractor anblysis requires several times as many 
cases as vapiablei to provide stftble results, onlv a' few more than 
30 statemehts could bte fac toped at a time, ■ Two grbups were selec- 
ted, one pf 36 statements and pne of 35, based pn* earlier factdr 
analyses. The statements selected were thoseimost highly -related 
to each. of the factors identified earlier. Although a few state- 
, merits were common to bbth, the; two groups represented BssentiallV 
difrefBRt groups of criteria. 

Frbni prior analyses, each group o'f statements was expected 
^0 iform six factors at the rooat. The- analyses bore out; that 
^pectat lon, wllh the sixth factor! at ^^t^ The 
relationships among six factors 'from odd^year staff reports and six 
from even-year feports appear in Table 19. / ' ,^ " ; 
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TABLE 19 ' 



Rgl at l-onshlps Among Fadtora for Qdd^nd ts/en Yeafs 
-Fattori frofh 36 statements . 
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Notesi 1* - Tha figures In tHg bodf of the^t^le can be Inttr^ 
; ^ as eorrelatiCft coefficients* , \y : V 

Decimal points- have been omittadv " . ' %j 
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^-^;;.;.,Qr^,|^e 36 s*«teme/its that provided the factors for ithe- top 
:^W&l(:fof^J^tilM '^ Six factora for *even and for odd' yeiri 'are.. 

; higrtiy 'conaisttfi They, show carrelations with . each other from . 86 
only-deviation from high, values in the diagonal and 
y Bluest: in the. ofr-diagoniia erftriea ifl the table occurs through 
„, Athe reveMal in prder tif Factor 3 and Those 36 atatemehts were- 



•thereforeviflpplied in; highly conslatertt ways 'from t969 to 1978. The ' ' .. 
. pbsftf vat iops related t© due ptQcess,: evmua admin- V 

-iatrat ive effecM^ other quajitfeg' thst cdlcieirie elds^^ 

y^ith tht- fi^ in mt^ning 

or fbcus frofn on© year tS inoth^r* . ; ' . ,j : 

: rbe io we r hal f ^ q f T ab le i 9 alidwi e 1 iqh 1 1 y; 1 owef ^tr^ 1^ 
^cl:«f rcut rel at iadships snwig the set of j? st^ttmi^ts for odd and t i ; 

even! ytars / wW Ie >he laat three factirs/are^^ : ; 

, : ■ . ■ . ■ ■ w'-, - : ..:t.-r- . . . vi, .. , ' . A . 

cpttelalLoH^^XM^^ from,;,a^iq ,.^4, tf|e correlations afnonq. mv-, '^^ 

: the: first trtree factt^^r^ ot . 77> and thesev IhrerfactSrg .'^^ 

are. not so cleaily defined. Tactor 1 for the odd years, which, is ' ' 

^concerned with due proceM a^d; includea some /of the sanie ^ 'Mi 

;;rfs ra^.tQr- 1 fpr the gfOMP ;or J6 statements it clasely;^ to - ■ 



both Factors 1 arid 2 for t evf?n yews. Simlfarlyr ractpt 1 f^^^^^ 
thB^-even years is highly, related to troth Factora 1 and 3 of thfcodd 



together^nd to bt jointiy sbsent in DE.AE staff dtscriptipfiB of 
acortditing, agar^ The high relatiDnahips In Table 19 show that^ 

the composition of triost groups of mutually "ohcur.rlng statements 
was quite cunstan|;. The same relationships among statements i* the 
same Ihterpretat ions of the discrlminaMng criteria, were there fore 
reflected in the KAE staff descrlptipna of agancies regardless of 
the year in ^^ich the descriptions were written. . 

The minor depar ture^ from consistency tfiat reduced the 
agreament among the firstjthree . factore of the 35*ststement gro^fi 
illustrate the nature of the cons^isteney. that produced the high 
values in the rest of Tab^le 19# Factor^ 1 Tor the odd years is 
defined by eight statements related to appea,! prpce'durej^v^due 
prqcesb, protection against conflict of interestt and concern for 
tKe interests of air constituencies. Fo^ the even years. Factor 1 
is related to appeal procedures and Factor 2 to due process, 
con fl ict of interest , and concern for all cpnstituerici.es* At times 
the d ist inct ions among those concepts were apparently blurred, or 
different interpretaiionp at different tim^s may have shifted thglr 
meaning* In general, thoughy the distincticns amon^ agenclea 
described in the staff reoorts remained quite constant , ^\ M- 

Finding that the agency character Vst i^s described if; staff 
reports formed consistent patterns of relationships is )^ot 
equivalent i ihawever j to findifig that the process thrniflh whic h 
recognition decisions'are reached is cdnsistent enbugh to pro Juc^ 
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the 6am4 dec iaians on* rgpeated appl icst ions o f the process to the 
same ageneiBs. That hypothe|^ic4i sftuat Ion cpnTOt be observed* 

i'Nevf rtNeleas , i f the in format ion entering^ reportx:^^ere 
>io t re I i ab 1^ Qb y rved and reoord^d t the HkelihoQd would be low 
that fa^Cori bisedr on two independent stts of observatlohs wogld 

' show the dtgre^ qf cof^ruenca displayed in Table }%y The results 
thergrora support s conelusion that the information on which 
dteclsions are based is reJiable/ Whetiier the infarmatlon and the 

^ decisions are correct or not is an issue of validity. / 

Bee^yBe some statements were.aommon to bdth, two ^et^ of 
r>factors-inVluded only about 60 ststertierits^ Yet th^ie co^nperat ively 
f^w sldtfmenti were the ones that provided ^ha qnfcatest diserimina*^ 
; tion smonq aqenp las, and therefore, had the greatest influenoe on 
. th^/ i^ae isions , If they ri?preaentad reliable information, a liaj or 
part of the information on; which the decisions rest is reljuable* 



; rhree indirect bu Cdfiipiemfintary approaches wefe used to 
;^eviituate the rel iabiUty or the rieoln^ 

K Aearedlti^ig agency directors provided Judgments of the 
obs^^vabU Hy of many of the agency chatsifteristics on wh;ich 
the recbqnltioh d^e tsions rest. Of 100 sfcekem&citi, ?P 
Judgac} to be reasonably obiervable*^ fefercif^ t^ 



7 ^ 



■ ' ■ .•' .•■166 ■ ■■■ ■• ' ■ " v; ■ ^. 

■ V / • ■ .. .r \ . . . ": - \ . s^. / ' ' .. . " : ' . ^ 

qiiali ties judged sbsecy^la dtait, tn generalf.: witH =the 

ef fecliventpelDf an agtftcy\i prpcedurMi^ while the obsSti^bii 



stater^nt a concerned the e^istefic^ of partlculir praeedurea* Thiji 
iS; |kn Dbv t©u8 kind of dlst inetlori, but It p0nta up the desirsbil- 
ity of develdping at least partial standiirdB for evalustinfl the. 



eff tee tlvenMs of eyalust l6n prpcidurtsj^;. tbe fairness of d^rtaioriif 
^he adequaiAv of mn oqitncy*^ staff i and bther Critical but subtle 

U th^ougrt^ most o f th# qu^litl^i judged difficult to observe 
^erfe^ sj^atiprod Ji'^dng th^ v^i^idu^/eriterlSs and ^jfieny nere balaffetd 
by other- more easily observed attributes that re ferr^fto the sipe 
cei^terian V th^ isfcatements In a general gfA^ ^^latftd;^^ variom^ 
Bspeeiu df an agency's integrity Mere almos1t^^n\l fofml 
dVfrie||it to Dbserve, Cpupltng joygments of importance wit h^ 
judqspents of observ^ility ^howfd thiie' qualities and a few crthers 
asEoitat^d with fivfe eriteria to be both important, and difficult tc^ 
observe. 1hi*. criteria rel^a to t^e accuracy and fairness ©T^ 
accred iting declaions, .adm inistrative and staff rf fectiVeneas, 
f ihane i al aoundneBSi^ autonomy , and prevent Ion of confl irt of 
.interest ^vn Ihe.fivo that most need further developtent If they 
are? to be rel i^bjy sppUed, . — 

2 Assessment b f t he deg ree to which the Inforrriit ion 
in the OEAE staff reporbs t*as a rel lablft statament pF agency^ 
character titles produced miMcd results^ A direct meMure of 



u^t*^e^;n Xfxi^ t^o i^ourc^s qf- inforii^alion could eitner r^hon tn^rij la be 
^r?riU'aclH of tr^^ *?rt;i5^nt ^ludy Of su^jg-e^it rufU^c^r strfpa'to imuta 
/!;^-f-^:5^-!;Tey.<^n- ci^r:fSinq u^o spe<T iT ^ ca t u^^^^s inv Ih^ .m format. 1 on v 

pfr^i^r^tion Of he^' ^ul.1^cnn^^^v -^^ari \l\j}f report). " 

X- ^'ot- tr:e JC-^ear period Ua^^ to Jt78# tne qf^nerai 

M-ii^#^rprct:rTj .f,ro^? an?* ^c*air la the Inttrpr&t^t ions did not- 

^tl)« pfocj^^s^ w^fe sm?in ■ enoyqh . in'">e|at ion to the total amnunt of 

^infor^BUo^ irtvoived th^it r-ajo: ^ncansisHenc les or depar lures . fro-i^ 

■'^ ^ ' . • ■ . - . ■ . 

,i^plMUsr^e4 stanvS^rcii^ «^re r/«r^, in short* the ^iqencies ri^coani/ed 

kind!* of difff^r^ncp^^ in 197B th5it/th«?y sihawed in r^^ifiier 

%l% qeri^ral ?5^p€ets, tr^e r«^cognit i;^n system Q^eratps' 
ro.liai^H^^ ■ Hq^it of hire r i i.^t ^ci^ cij^^rl^^ ^nuyq^-/ (j^? f i ned any ' 

decisions ri!arhf?v1 are cc^para&lt = .far similar ififincies end ffo^» 




,?iir to fi'Air. ^ fi'^ of tne critprUi, thoujn, fp^er to tne e^rects 
of flr^ aqencv's proc&dufes rathi?r tnan to ^et^er good operatir^j 
proc?£fnur*?s\ arff fallo^vri. S^^ np^iication tnes^ criteria, ^hich 
nte reifitfc'^ to fhirnei-n%, int^qrity, orgsni^at iDnal strength, and 
r f f t 1 V enf^as of evalunt prccedures^ NO'^ld be s t renq thened 
tnrouqh det^U^ ^bj?ly to clarify their de^rifut ions, impl i^rat ions ^ 

to nf^nti^>^tnp incid^ntQ likffl;. to interfere wUn iffpsrtisl 
u^cin^ enl^ or rair ^application of ^t^sndardts and to minimise tr^c? 
^r.^^Qii^^^"^ ^: ^ *heir orcur r<^<^'"^ . erit^ri^ ^or ^^fectiv^ 

c^djj^^i lon/il n ! uJf t; i on ! tr;??n- &e 35 rfrliafoiy obtiftrvod tne 



vin. viUDiry 

fno validity of a proceBS iS the fjegree of cranfiUftrirp m t,r;e 
inferences dr^wn from th^t proc^^H^ that r^?st5 r i an e-f-pirical 
ba^e. Valicuv IS attached not to the process directly, but ia the 

proce^in r&coqnizer: aqencies provjr^^ nncuratf? judn^ent^ of 

t^o t^O:^r ai lonai quality of th^ proqrams or i nst i tut ions tr.f?y 
^v3ji;^t^ . I- t^e s&nenee of indisputnuiy correct knowledgf of t^itf 
evaluative accufacx of the rouqhiy 100 accredit mq and state 
VipprGvai aqencies that havR '^otjqht reeoqnxtion* indirect evidence 
rj^f tnf! f^^fit of the infijrencew drav^ from th^ recognition process 

Inr#r k infin of evidence of kU..)/ were providisd, Tn^^ first 
^n- an empirical differentiation of sgf^cies an the b^^i:, of Ifm 
xir^l&ili*<] i n for Tiit io/i MtiDul tnem that entered the r<*CDQnition 
riecisionn, if thrr /reror^hit ion pfoc^sii fJ i §c r i^^ inatP '5 iipprcpr l^l 1 1 y 
a^O'^q anient ae?^, * hofif rernQni^ed ahauici be t^^pit icuWy 6i(t^t&r\^ 
tiPible ftar thos!? denied r^^aqnition. further, the nature of th^ 

r^r A.qni/^d as knawI^^SrjMbl^ , oe important find iippropr igie dist ifjc- 
tions bet^en ^onc ies^that are ^ffeeti^t iind Ihooej-that arcr not 
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tm- ffvi^iuation of tducational quality. Botn th^ne concJitions were 
met ^ alti^'jOuqh so man^ elements enter both the accrediting and 
tvcoqnii ion procesaes that some inferencfs dra^ fror^ the recog-' 
rMiinn decisions are n are fuily just i Tied than others* 

^ ne ripcond source of evidence the judgments of difectora 
erf? rfit inq and state approval aqencit^s ond other knowledqealp - p 
pf':^or\h to ihff importance of the varioim eleffienU^ o f inrormat ion 
nn%ociBt^*d s^it^ tne recognU ion criteria, Uie^e judgments not only 
irulir: rr^r npp ropr lateness of tne disti^rtions amnnn ^|enri^^, 

nUo prDvidf>1 evidence of t^^c clarity of the mterpretab^ns 
nii^^hed io individual criteria. 

\ 

third ^qurc0 of evidence was a sot of simulated jyd^iT^enis 
hv 3 ^idelv diversified group of inrormed persons, inciudinq agency 
(iitetlQpn, who «75ade their ow recwnitinn decisionE baped on the 
n/ime informatian that had entcmd the Bclml dec iSionB/made by th^ 
''-.^-^.^n^inni^T nf CdurHtion. Whilf* mani of the rv idene<? of validity 
xind re I I an ! ily relflt^d to camponef ^ of th^ r^cagnition 
procesi,, tt;i75e Judqmcnto addresseri morit: of U^e proee^^ a?^ ^» 



in Ch^^ter V, the ways in iNhieh aqenciM were d if ferent ia 
by thp DrAf BtafX repofts were described throtflh mull WiMn^ionsl 



seniinq of th^ aqency denser ipt ion^ » In q^neral , tMe aqencies 
denied recoanition were Hidely OMpara* . d from the ather ogeneies. 
Iht-, analyses dp^icribcd involved two random samples of Bgenc'i^s 
i^hich inciudod CTDmparritivel > small numberci of oqencies thai lipa 

tionn mn^i^ in v^f^ W:M st^iff reportr mmq all aq&ncic:^ - a total 

qroyp. But almost half of the ^lecisions mucie aft^r P , ^len 

Ihf? ^orr!*^^\er iUr la ^rt Iwim used, have been for rerm^nu Vf r 

'«ur ytfars. Each o.f the othfff fciur lypea af docioion, rrocn i'sn : 

to rtfCoqniLion for tt^re*» ycrgrs, accDunty for froiit iO to 1^ ^^rcen^ 

fjf thc^ d^cisiyns since 1974, Qceaune of uneven distributr 

of a^r^nion^^ ih(^ distinnLions Msociatfrd ^it? ^ *^f^rent ^iec isiot 

■ ■ \ ^ . 
#^ere nat cie^^ in the biirli#fr analyses ^t^ might Ha.>i 

hwn. - ^ " . ^ • • ' 

ali th^ ciffeislqns nil h UAnI qrom^ Bmple of iO OCAt: stuff 

r^noft^ wfiiti for^^fi, cansi^lr wtirciiy of reqionsJ or .spteiai U#d 
m^r^cies r^cmr^l^^4 or dr^nif; -^^rt^nit idh m 1975- or 3o^i?r. T^n 
"--4^wcien i*f*r«* rnndomly selected td ^r^s^nt rA th^ fiv^ type^!^ 

or dif*ei!iirins>^-i:fi^ni/il, nmj. recoqnit ion f?^r otc* Uti* Inrm, of four 
-velars* -. fhe mul t id Mnec^n' sei^nnq (hDS) procesii df&critt0fr 
ear I I* f^as appl M>d lu ihi& nn^^pl^. of SO ^laff rf^ports . ' the 




llqoff Ojff«rr«?AtiHiion uf jijHrt^iiia t^mym\ d\(fmmi tectKjniliwi tinoisioni. (Syib^k' | 
fe V feprcsisil rwc^nitton Mir unc, li*0| three, or fmir yy'irs, nmj dfiniai nf reeori ucm.j^ 



J He t%*o* dimensional plot in riqyrc 4 reprefents the simllciri- 

tim B-mnq aq^nci^s an cj&wrib«?d in the staff reports* The smsiler 

inr distiincg betv*^#n anv two symhols, the more similar ..^ere those 

ifw ^q^n^iws in th^ OtAC staff descr ipt ions, SimUifity .^w measured 

of thff nu^b^r of descriptive statements that weft . corrjmQn 

to ;h?v pair of acjynci A^i^^^hch^b i^ith eor^^parfit ively lerqe nunber^ 

0" desuri.pt 1 we terf^^s in cotmon would be CAp^ated to appear close to 

o:ic^ cnr.cr in Miufc? H tnt recognU ion dee is ionii ri^j iably 

rcffU^ct difietencen in ©qe^cy charactorist i^s, different dGCislons 

f?nau|d apR^^r d i f f^r^nt req iom of Figure 4, If the deeisionfi 

vat id, Ih^ ^aff^renee^ ^ill be wso^iatgd wi th the^ merit 

^ 0/ im mmci^n' juK^min of educational quality. 
^. . ■ j 

Host of tm nqenci^n are clustered to the lift of centtr in 

Ue, fiqyre. If^^rteen ^aneiei scatter verticaily and to the riyht 

of- t|\f? fT^ajor qtoufi. Jhos^ 13 include seven of. tne 10 danied 

rf eofinit ion , two recoqnUeri for ong year, t**o recoqnizedl for ti^o 

v^arfM ^^^^ reeogni^i^d for Ihr©^ years, and one for four* The 

■mmvit^^ tmi dU ?>f,..ma»t, fr^^ the central --qroi^ mve ^learlv those ' 

idrrit^fiftd in ihe r^eMnition prmfem d^fieii^nt* Horeoveri lhe> 

■^^r^? .not df?fieion^ sn simjilar w^ys; the y^. do nst farm qroups^of 

^^mtm *#ith .co<^on daficl^r^iM/ fhe, iqencies .denied rec^rrition' 

^it(fmf\ m^rwi thrmwlveg much more widoly than tht ^ene res that 

■form the miiln qreyp.^ Nine <tf t^ft lO^cncics rcc^ni^ed for four/ 

y^brs form 3 stiU tifjhter grow at the center of the main group. 



Imo par^ilel lines have Qm6 drawn in Figure 4 that aeparale 
it, into tfirf»e regions. Ihey slant upferard to the left, estahUihing ' 
a r<?qian to the right and thn tap oT the^ fU^ur^ that ineludes sevftn 
fiqencieSs all denied reeoqnjt ion , Betwften the lintjs is a 
region of transition where 7 ef 13 agtfnu:ieti were recc^ni^ed for 
only one y^ar. one^os dtmied recoqnit ion, and f ,^ wt*r^ recoqnizftd, 
for or threr yt^ars* lo tne left of both- lints are 30 of the 
antrne - tho ten agencies that inf^re rec^jniifeu for *Qyf yeors* 
eiqht of the ten mmcim rfecoqni^ed for throe years, seven of the 
t^n tecoqnitmi for twd yeara, thrge of the ten recognised for one 
yMr , and twii of thoie denied recc^nition* 

!ht arrsnqement of aqencies m figure 4 showS a cleor sepora* 
tion of most of the aqenciee denied recognition from the othefs^ 
and the location of most of those^ reeognifed fa r,, one year is in a 
region between thg danied sgencies and the -rest of the agencies* 
but one of the agencies recoqniMd for four years are tightly 
clustered m & smell regisn, Onl^ agencies recognised for t^ or 
thripjr^ v^*iirs--"reQumfthly thoiie with some deficiencies « none of them 
^f*^eftr*are not clearly different lated, for that degree of clarity 
of siepurotion to occur among ^encies g iv^n c'if ferent recognAtion 
dcftisions from ' to 1978» the r^Hat>iUt> of the decision 
process had La be at least m^epatelv high. Leek of consistent • 
or error in the procesSi would hive produced . imilsr decisions for 
agencnes that did not show itmilar ctiaraeteristics, and agencies 



with trig* sum© recoqnit ion •jtalus would have been sc^^ttcfred 
throiifinoul the fuiure* ; 

Ifm m^^^uvu of s imi 1 nr il thai produced tJ^e pvitterri of 
f iqurt. nau i>am^ aer|rep of error unavpid?^ly byiit into it, as ^m^ 

^ntininly iinder?»!it irnat^^ of Ihe^ truG similarities, actunl 

tHer<Kor«? lik^Iv to qr^ifttr than is*shown'm rigure 4, » 

whH^ reliabilili in the rejcd^nitidn process wa:^ rt^quired tn 
pr^dijrf*' fha flA9^ri^ir^nl ion. in ^ xqut^ fi , validity lies w|th the 
m^?iininq isniit can attached to tf i f ferences b^twen ^enciea at 
difr^rant iocHliOn^ in ih^ fiyufi** Ini^L meaning' Csin be d^lermirwd 
Uy iderairvinq tht ^lolememo that utb ^hargct^rist ic of ageneirrs 

aqtncv chjiract^rist ics, (he f ?rlhi?r an ^mcy is lairated to thr* 
right- m fiqwr^ 4, thn ^ort U ^aa to hiive hnd wak evalu- 

ation pracgdufM, to Ha^0 Hiid a qov**rning bmrd mi fully r^prte^^-- 
tative of nil enn^t ) t uenr to or th^ ggnerni pyblic# and to h3V^ 
left some 1 about its impart lai ily* As aqencids appear fartHer 
to the lef^-f they were m^ire liktly to be strong in Ihom^ thrue 



oralis. In Table 20 are the statements that tended to chaj^scter iie 



r^ef^cies rut opposite gild's of the horijontai dimena; 



Char&cl«^r^3tic[> of ih^ left Cnd 
Ev.a luat i ve procedurgs . ' 

120. n*e acj^r^y use^ co^^pe^^ni pctopl^? 

on visit ifw) lea^a. 

.■^ 

129, The agency prtp^res visUinq 



Ch^Tact^rist'ic:^ of "thg* Riqnt tn<i 



?^J* Self-^atudy proeeduret are 

dfifieifrnt in that they fio not 
provide for suf ficic*ntly 
critical or snelytic ass^ ss« 
mr*r\i of ^trtngths and weakneeses, 

'2W, Th^ ^gncy does not\ pfovid^ 
iuffieient preparation or 

instructions Id the viSUirHj i^m 

279, Tht aganoy'^ written .or cmsulto- 
tiv0 guidance to rnstitutii^ns 
bcrore a site visit is iCidequate 
Qf nonexistent. 

5-59* Informatiari about the sgency's 
mothad or errwllveness in 
riBs&3??ing tHe rel labi I ily . or 
validity of its standsfrds is 
misusing gr inconeiusive* 



.280* ^ .1 rKjsfiey tiikgrs into fiecount tm 
i '^^rest^^rdi riciiMs of some or 
Hi I of tne foHoWinig eonstituen*- 
ci^fif stuiffnlfi^ puBUc, faculty f 
^ institutidM gr pra^rs^^s* end 
oeeupat lon^l^ proftsaional und 



iZ7^ Jm aqency does not provide for 
or currcntiy have public r^pr^- 
s^ntativts on its po Ucy/d w is ion j 



/ 



■ ' . ...^ Of F^mur*^ i = , ^ ■ \ ■ . 

Cncir^*ripris<:jc- thtf left E'hd - Ch^uac^^^r^si ie^ r^f tue R^nhf FnH 

H^^rrpcin^ Hvi?nes^s (contMJ ^'-.-'^ ' 

" .:. ' ' ^ ' ^• 

"?3i. aqmrv hijs .public rs^rMtnta- or mart etnriMt u^rtc i^ij riri;^ 

into n^^^eouf^t . ^ ^ . ' = ■ . \ 

^ ' \ . \. . 

m:npp^Tiunii% to' react to . . 

^ qroyps ^fiP reprasented on the ' ^ ^ ■ . 

lhe"^^ney^ii sian^^fclfi rire cIm- 6i0. Some doyl^t c?xists as- te ^^ther 

fa^r and rtUable treatme/'^^ to. , and o^^jwtivcr slit ion^^ 
SRfit aut rgns Of progfaM* ti^a^xmn^ ■ 

J -rtml .with dye* noU^ ■ , 5 ■ ■ , 



lha aq^n^^v'^s pgbliahcd proeocJurto 
are contfl3len| wiWv its pi^OMduros^ 

fair and consislwl sppliMt^OT 
of its siandards. 
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1A3U 20 

of fiqure ^ 

CfuifBctti ABt ICS of tha Iftft tnU 

710, T^e «trrcy performs no fumtiori 

■ of m ir>HtUutic^;. 

"^j/ H^^bt^. 3^ of the d&cHlQrt'-mBkinq ' ■ 

.bodx-dg not participaUMn 

afTMltng institatloni wUH 
Hhteh they' Btn afTllinted. 



wU^^^r ^t^ti^M^^uis s ii^ adt.^l,4ai^ lu tUQ^e ^hwi^fi 'in ■ 1 able 

?y itat^munti^ Ustcrd sr^ t^^ot© that give' im most ommrent 
m^ctipiio'^ af tht difrerancf^ associated ^ith that dimension*' 
tmy ^ir^i qyil<? aimil^r in t^eir referenees to evaiyative pro* 

..,,di?cr ibfjci in Chapter V. In BMii ion^ tHf* agencis'^ to ihe right 
^l^^^^citf^ to thiB loft tended to nwredit ^ore ihjin Sn,- . 

■ • , ■ / - ■ ■ 

Aq^mcies f^om ii%m^ dtnii^d rMo^naidn, but aol as strof^ly as 
cjid th# hof ijrontal' dimensi^h. The ^iq^nciei at irm top Here smaiU 



um^r ^n^OdO, nnd prompted doubU nt>Oi^i ihi* m^qumy of th^ir 
^luffii^ '-tnr ^ir^nqths ^ere hi haw ui^^ . c leaf orc?ani j^ation^l 

pUinis, f^r tnp ravi^^ ^md t^si%ion of slandOTd^, and for ipi: 
^r^llt^lton fif mol icml lom ^ar ncc-redM ?u inn. Thi^^^ ^^i^n^^^^^^i 
nal m stf^ous t?^a?^*'''ar >hc? hor^^^ntal dinwaion' ancl do not fonn 

* _ 

iCk^^T-^ ing low end of ine^vtrtical diitiGnsion iendcd 

fr^^i, **^r^ adMu^tf?lv finiinrf^rH /iah nk^|y f t} H^iv^. ?*^Dre t^^^ '10 yoar^ 
ci^ ;j iis an flecrediting #qency* fhe dominint aspects .oT 

^ hori^i^^ntal dimension* i^h^ch most ei^fly dijieri^tiinated' 
^^twf^^^fr ^^i^^f. j^d and recogfu^ed ^^f^cies, 4warnarai€d :^.ubst^fiUw- 
is^y^a ^lyc^i an the st r&rif|!.h of tm fir|^^firy ' Vvaluatiw procedures 
and ih^^ i^porti^ilR^ ftf^ts actions, t'jQ vertical dimension 
tiff jn^^r QU4)I it ies--li^U3tions si^e^. ws^alth. anc^ 

^i^per lenee*- tria| miqht nQt in themseHss h^v© led to d^Ajol of 



Tne inifd t1 ^^i?ns ion riot ^;h4^n, r^presenicrd less clenfcut 

It s^nSf resisted to ttic? re. rt/senlat :LOn on the governif^ board of 

all constituencies and im general public* it also reriec'fd 
■ ■ ' ( "^-^ 

^dcUtiyfial scattt/red altritouttts, botn positive andfn&galive, at 
eii?^er It did not diTCri^jnc^te among different rocoqnilion 

l^i^ icom of. pi^g^^jws ■ ^ ' . 

i ■ ■ , ^ ^ ^ ^ . _ ■ 

A dg-fiir^ibl^ qua! ily of the inrormal Aon ifi Tlqure i^Mch 

d .co^tr itiui^ ta tKi? vsHdity. of the rtcoqfiit ion iprbctss ^ 

bv:^'rie*:sn:^lr3t inn zuff icwnt qtropfe^' in the inftfr^.^Hon^ on w^ieH 

A tot^l of S!>^iiiatt^ments di^iminsti^^^^^ agencies ^lonq gna or 

Mi^f otter ti^ aimenaians. ^M^if ataUmenl% were ssMc ialed 

»with rill nu! 7 of th^' 4a efiterig. Those 7 require that tht 
m|*?nc> .St^ndafds that the viisUing team 

mQi wi^h siiiji^fr?:ntn, fMul ty t and s^ff ibSii Ji that institution 
or program, be qivt?n cko'Y of the visiti ieam*s rtpert (qJiiUj 
*hat thf* ^*i^r rii^teut 1 DfriCf^r Ha^s an opportunity to respond 
loathe report (b3iv)i iHat spf?cific reasoni be giveh for advtrst 
dinner isions ^blviih th^t accreditation nat be ehangtd riuru<g m 
uppeni ibiv n t A.) i lifid th;«t tne eraef Kxeeutiv^ ofMcer be given 
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a written statement of. the appeal decisiori (b3viiiC)/' The failure 
of these seven criteria to discriminate amonq ^encies y^BS probably 
due to the low number of agencies that iFailed to meet them. 
Deficiencies werr found in about 10 percent of the aqencies^ with 
respect to the criteria dealing with the meeting of s^isiting 
teams with all campus groups, and the submission of the visiting 
team*s report to the institution or''^ program . beficiencies in 
^±he other five criteria were found in from 4 to 7 perceni of the- 



agencies. The favorable statements related to these seveft\criteria 




tended, correspondingly^ to be quite frequent/ With little vai 

■ ' ' ^ ' I " ^ ^ ^ ^ ' ' 

tion, they could not discriminate clearly among agencies. Thpt 

does not discredit their value but only reflects the fact that most 

aqencies^met them without difficulty. 

The representation of 37 of ^he 44 criteria on the t\vo 
dimensions aldnq which agencies were discriminated may or "may not 
be considered adequate, Some^^ay hold^^that all 44 should con- 
tribute. Failure to discriminate, however^ does not represent a 
failure to contribute if, as was the;case, almost all agencies met 
the nondiscriminating criteri.a% The 37 criteria associated with 
the two dimensions of Figure 4 provide a moderately good represent 
tatlon of .tKe 44 criteria in th« information on which decisions 
are based. Further,, the decisions were not dominated'by a few 
overpowering criteria. Although some "^riteria were more influen«: 
tial than .others--those cqncernsd with^ evaluative procedures, 
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respons 1 V en'ess to-^various publics, and integrity--almost all 
criteria entered into the actual decisions to^ some extent. 



Overalls the pattern of decisions shown in Figure 4 indicates 
reliability in the clear separation of agencies about which 
difFerent recognition decisions were made, .Validity in the recog- 
nition process is indicated by the nature of the agency character- 
istics associated with thei-r location in the figure. The two 

dimensions, particularly the first, distinguish appropriately 

\ ■ ^ - - 

' between, agenci'es recognizsd and denied recognition, 

IMPORTANCE OF. THE DISCRIMINATING STATEMENTS " 

Another kind of evidence that contributes to validity can be 
derived from Figure 4 and from the jud^ents. of impDrtance of 
various attributes to an agenfiy's effectiveness in determining 
educational quality. ^Confidence in the inferences drawn from 

- m ' ' ' ' . . 

recog ni t ion would 'be strengthened if the qualities that most 
sharply discriminated the denied from the recognized agencies, were 
*alsb among the most important qualities. 

■ Of the 20 statements in ' Table 20 that contributed most 
to the discrimination of agencies along the horizontal dimensioni 
13^were included in the questionnaires on importar^e. Ten of those 
13 were in^.the top hal f with ^respect to Judgments of importance. 
-The discriminating statements that were low in importance were 
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those related to the represeht¥f lo'n "3 T tW^^^ public"ana 
other const ituencie's on^ agencies' governing bodies* Including 
public representatives in an agency's policy or df cision-making 
body was ranked. 7l3th among the 80 characteristics judged for 
importance by agency directprs. Even 'among the other judges^ who 
represented a wide varTdty o f viewpoints , that characteristic 
ranked 42nd in importance among 54. 

While publ icV^resentativt'^s on governing boards was widely 

considered of only moderate importance, having representatives of 

all ,^ighif|cant constituencies of the agency on the board might be 

expected /to attract more interesl\ It did, but barely, ranking 
'..=/' ■ ' .. - ^ 

67th arnong the 80 statements judged by the, agency directors. It 

was not among the statements judged by the other groups, 

■ i ■ . 

- . ■ ^' 

The direct representation of all interested groups on 

_ . • . ' J- ■ ^ ' \ 

governing board-p of agencies was not considered critical, but 

making the^agencles ^ policies and procedures publicly accessible^ 

was. The characteristic considered of highest importance by bath 

the agency- directors and the other groups of judges was that the 

aqency's standards be published. Other statements to assure that 

^ '-. ^ " ■ • 

governing boards would be attentive to the interests of various 

. ■-- 
constituencies were also high in importance. That faculty members 

and members of appropriate occupational and professional fields 

have access to the agency were two attributes considered highly 
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'.mporAant , but the indlusion^qf itudent viave.^ 
.rocess was considered somewhat less importtnt* The statement 
: 3ted to the solicitation of student viera ranked in the middle 
: 1 importance. 

Thus the presence of represtntatives of the genersl public on 
Qaverning boards is a characteristic of accrediting agencies that 
contributes substant iilly to recpgnition decisions even though 
several groups of knowledgsable persons judged' it as comparatively 
low in importance. In the judgment of many informsd persons, the 
comparatively influential role of that attribute in recognition 
decisions reduces the validity of the recognition pr^ess. 
The pertinent criterion is b2i, which states, "The agency or 
assoclatior includes represent atives .o f the public in its policy 
and decision-making bodies, or in an advisory or consultative, 
capacity that assures attention by the policy and decision^making 
bodies." None of the groups of informed persons contributing 
to- the study— agency directors, administrators and faculty members 
e^Kpcir ienced with accrediting, federsl and state officials, critics 
and scholars of accredit ing--thought that that attributi con- 
tributed' greitjly to an flgency's effectiveness in determining 
educational quality. 

Despite the questions raised about the effect of the criterion 

related to public representation on governing boards, the /decisions 

• ^ / / - \ 

generally reflected the criteria considered most important* Thus 
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denial of recognition did indicate deficiencies that most informed 
observers agreed detracted in important ways from an agcmcy's 
effectiveness* 

SIMULATED RECOGNITION DECI5IQNS 



The First evidence of validity deicribed^ above consisted of 
the degree -to which agencies receiving different recognition 
decisions could be discriminated with respect to two dimensiDQS 
that reflected most of the recognition .criteris. The second kind 
of evidence consisted oF expert judgments of the importance of the 
qualities that differentiated recognized agencies from those denied 
recognition. In general, the discriminating qualities were those 
judged highest in importance, A third ap^p^^h to the validity of 
a process intended to discriminate more effective from less effec- 
tive agencies is to compare the results of .the actual process 
with an independent process designed for the same purpose. Since 
carrying out a full alternative recognition process was not 
feasible, a simulstion was planned that would ask e^cpert judges to/ 
make their own recognition decisions about fictitious agencies ttjpt 
matched real agencies in critical characteristics, 

' ' ' ' ■- 

Each of 12 fictitious agencies was described in several 

paragraphs that included from 20 to 27 statements drawn from the 

list of statements appearing in "DEAE staff 'reports - Statements 
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vvtre combined that were" characteristic of agencies denied recognl- 
tion or awarded recognition for periods of from one to four years. 
Thus a description made up of 25. statements pl^s connecting phrases 
might ha^;^e included 8 to 12 statements characteristic of agencies 
recognized for two yeprs, plus 13 to 17 additional statements not 
associated with any particular decision. The 25 statements consti- 
tuted a realistic^ coherent description of an accrediting or 
stat.e approval agency recognized for a two-year period. F.pr - 
each dfecision eKcept recognition for four years , two composite ' 
descriptions were constructed from the discriminating statements , 

and from^ enough other statements applicable to any of the grdupsi - 

■jfc. ^ ^ _ 

to round out a realistic agency description. Four composites were 
. formed of agencies recognized for four years , making 'a total of 
12 detailed deicriptions..o f fictitious agencies. Each agency was . 
described entirely by statements tjiat were characteristic of 
agencies QWarded a particular decision or were equally character- 
istic of agencies of any recognition status 1 

The 12 cQmposite\ agency descriptions were examined by expert 
judges yyho then made their own decisions about whether the agency 
should be recognizea as a reliable authority with respect*- to ^ 
s educational quality and whether any doubUs about the agency equated 

. that should limit the period of recognition to parmit an early 
review. Thus USOE's redognition process was simulated by an 
r independent process undertaken by informed persons.. 
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About ,hair of those other than agency directors who had 
been asked earlier to judge the importance of accrediting agency 
characteristics were asked to partid-ipate in the siinulttion of 
the recognition decisions* They we're supplemented by .directors 
and cofnimission members of accrediting agencies, A total of 151 
persons--schooi and college faculty members and administratorB , 
federal and state officials concerned with accrediting, scholars 
and critics of accrediting, as well as agency directors anc 
comm iss ion e rs^-wer e asked to make recognition judgments about, tij 
fijctitious agencies. A total of 114-=75 percent of thos^*-^Red»- 
rpsponded, including from 60 to 91 percent of the different groups*' 
Although the. irtdividual groups, were too small to treat separately, 
the largest consist ing .o f only 23 people , the 1 1 4 persons who 
provided their judgments varied widely, with each one knowledgeable 
about accrediting from a pirticular perspective . The faculti 
members and administrators were, from large and small institu= 
tions, public and private, degree and nondegree. The agency 
directors were from regional and specialized agencies, large and 
small, institutional and programmatic* The government officials^, 
.scholars, ^nd critics also viewed accrediting and recognition from 
equrally var^led frames of reference. 

The results of the simulatef! decisions are shown in Table 21* 
The full quest lonnaiTe with the 12 fictitious agencies appears as 
Appendix/G. For eight of the fjntitldus sgencieSf the conseneus of 
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TABLE 21 ' 
Simulated Recognition Decisions 

Fictitious Agency Deciiion 

4 yrs 3 yrs 2 yrs 1 yr Denied 

1. The Advisory Board on ^ ^^^^^ ^ 

Ophthalmology. 14 ^ 23 8 4 



2. The Amf ficEn iCouncil on 

Internal Mediicine 71. 17 (l^ 



11; Regional Schools Accrediting 

Agency ^ (sp 19 8 

'12, Schooli of Computer Science ^^^..^ 

Advisory Counpil (p^ 20^ 12 



3, The American Council of 

Registered Tax Consultants 6 17 40 (28) 10 

s4, Jhe American Ff^deratiah of 

Speech Pathologists 25^ (46)., 21 

5* The Association for the 
Advancement of Automobile ^ 

Technoldgy^ ^ 3 10 41 (31} 15 

6« The Association of Campuses 

Without Walls 1^ 0 4 5 (90: 

7. Cdmmi ttee on Research in 
Medical Technology 12 4 4 

■ - ■ ^ ^ ^ : - ' ^ ' I . / 

8. The Council on Corporate ^ ^ ■ 
Administration .46 2^5 J 

9. The National Association of 
Counselors for the Elderly 1 0 1 8 (90^ 

10, The Paramedical Society of 

America ' : ^ ' . (48) 29 



Note ; Flgures..ara percentages based on 114 Judges, The 
act'Jal decliipns for the agencies from which the 
composite descriptions were drawn are circled. 



X. 



/ 

/ 



189 ■ . ^ 



the judges' decisions coinci^gd with the actuad deci«ions* For 
of the other four, the judges' consensus wis recognition for tv^o 
years while the actual decision had been recognition for one year* 
The Judges awarded recognition Tor four years to the final two 
agencies! the actual decision in both casis had been recognition 
_fcDr two years. In each of the four cases where the judges dis- 
agreed with the actual decision, their Judgment was more lenient. 

The easiest decisions—those on which agreement ^ was highest^ 
both among the Judges as a group and between the judges and the 
actual dtec isions--were those for denial. The inadequate agencies 
stood ,out sharply' from the others in this simul at ion ' o f the 
recognition process Just as they did in the multidimer/sional 
scaling analyses of actual decisions. The next easiesi decisions, 
as would be expected, were racognit ion for the full four years. 
The most amb fguous .were the ' intermed iate decisions'--recogniti'on 
limited to from one to ^hree years. 

Agreement of Judges with Actual Decisions * 



t 



\ 



A measure of agreement between|two sets- of judgments that 
classify objects into categories that ihave an inherent orQerj-such 
as" the five kinds of recognition decisions, is ganma (Gbodman 4 
Kruskal, 1954). Its range is from -t. to +1, with a value of zero 
indicating no association between the two sets of ordered Judg- 
mtnts, -^Gamma was calculated for the issociation between .the 
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modal decision of the judges and. the actual decision for all' 12 
fictitious agencies as a group* The calculated value of .68 
indicates that if a pair of agencies having different actual 
decisions were selected at random, the probability that the judges' 
decisions would place the two agencies in the same relative order 
as their actual decision is greater by ^68 than the probability 
that the judges* order would be the reverse of the actuil order. 
While this giv3o a quantitative measure to the degree of aasocia- 
tion between the Judges'* decisions and the actual, decisions , the^ 
most informative indication of that association is in the 12 
distributions of judgments sh^wn in Table 21. Two of the 12 were 
off the mark I the other 10 weire quite close. The percentages i of 



Table 21 show the frequency v^^ith which the Advisory Committee;'s 
decisions would be |supported by a fairly larger highly diverse 
group of people who are knowledqeable about accrediting. 



Having made a recognition decision about on agency, the judges 
then indicated the five characteristics that were most inriuential 
in their decisions. Tallies of the characteristics that were 
most frequently judged- as influential showed that concerns for 
fairness or impartiality ^redominirted as controlling factors in the 
decisions. The sin statements listed below appeared a total of ten 
times in the descriptions of fictitious agencies. Nine of those 
ten times they were judged among the |ivB^^most infiuential state- 
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ments by at least half of the judges. The median valuer over all 
ten appeRrances of the statemBnts below, for the percentage of 
judqiee who pi peed them among the most important characteristics, 
was 66 percent . 

The first four of the Gharabterist ice listed below were 
influential in decisions to recognize an agency for .three or 
four years. The last two were influential in decisions to deny 
recognition or^ limit the period of recognition to two years or 
less* All six statfnents are related, to fairness^ impartiality, 
or autonomy* 



594, Clearly estabdi^shed procedures insure 
impartia^l evaluations and judgments* 

j 411* The agency's standards are cl&ar ind 
detailed enoug^h to assure jfair and 
reliable*^ treatment to institutions or 
programs * ^1 

590* The agenc\y's published prDcedures are 
fairly spp'^iedi its actions are ionsistent 
^ with its: procedures. ^ | 

710. The agency \performs no functi'on Incon- 
jistent with\ the exercise of i|ndependent 
judgment about the educational Iquality of 
an institution* = j 

729. The agency -s relationship with jits parent 
- \ organizntion raises questions of autonomy* 

610. Some doubt exists as to whether the. agency 
ensures impartial \^and objective evaluations 
. and decisions * " ^ 
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Six of the twelve descriptions of fictitious agencies included 

a statement about t^he adequacy of its staff, twice in favorable 

terms and four times in unfavDrable ones. Those statements were 

judgf?d to be among the five most influential by from 15 to 69 

I 

percent of the judqess with a median value of 57 percent* Thus 
they tended to be considered among the most important by s^^ajority 
of the judges* * ' 

Several statements about the agency's procedures for evalu- 
ating or improving its standards appeared 11 times, 8 times in a 
positive, sense among agencies recommended for three- or four-year 



recoqrti\.idn decisions and three times in a negativn sense, in^i= 
eating a deficiency in that area, among agencies recommended for 

^denial or for a one-^^or ^two-year period of recognition* These 
statements were considered among the five most influential by from 

.21 to S8 perceht of the judges, with a median of 43 percent* 

The characteristic m.ost consistentl y judged as influential 
in unfavorable decisions was a deficiency in an agency-s self- 
study procedures* Although that statement appeared in only two of 
the descriptions^ it was both ti^es the one^most frequently Judged 
among the five most influentialN^-by 81 an^. 72 percent of the 
judges, respectively, ' ■ . \ 



Statements that appeared frequently in the twelve descrip- 
tions, but that were notably infrequent among the most influential 
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five, dealt with the monitoring of ethic/al practices and broid 
representation on the governing board. "The median frequency of 
appearance of those two types of statements among the -five most 
influential in 'any dftcision was i respectively , 14 and 24 percent. 
Tfie range was 3 to 32 percent for the monitoring—ofr ethi^cal 
practices in recruiting, advertising, tuition refunds, plscement, 
/ifHj nondiscriminatory practices in admissions. For broad repreien- 
I'.iDn on the governing board, including a concern for public, 
r s r^^^^^'^tat i ves , the range was from ll ^^to 47 percent among the five 
; inriupntial aqency attributes. = * 

f.vpry statement among the 285 used to describe the 12 agen- 
r:f^5, even those as innocuous as, "The agency is applying for 
r-^'p^nl of recognition," or "The agency approves.,programs. through- 

* r,p Un ited States ■was judged among the f ive most influential 
: . n t. l^a^t one person . Yet only half of the 285 'Statements were 

; ! irpj in the five most influential by more than\ 15 percent of the 

\ ■ ^ ^ ■ - • ■ "■ 

; .'iP5, The median values of 66, 57, and 43 percent for statements 
ill i!pj, respectively, to fairness, adequacy of the staff, and 
' . il ir.tion and improvement of standards ire thirefore noteworthy. 
1 n triree areas and deficient self-studies were clearly the 
if r^in int considerations in the decisions reached by the group of 



• NCE ON RECDGNItinN CRITERIA 

/ The resul'^.of the simulated •recognition decis were 
discussed and.exganded at a one-and^a-hslf-day- confirence by 44 of 



the persons who had participated in the simulated deoistons. 
44 participants included all the varied points^of view that were 
represtnted In the 15l\peraons wh^ had contributed their judgments 
in the simulation process. Alternately in small groups and in the. 
total groups/the assembled persons examined- in Free- flowing 
discussion's why some agency attributes were critical and others 
less ^ in deciding on the.wrierits of an agency's judgments of 
edijj^tional quality. The dlvprsity of the group ensured the air ing^ 
0;f divergent points of view, . The smallsnumber of representatives 
■of\^any particular groups though^^ prevented different, views from 
beih^ attributable to any identifiable group> ;The diversity of 
Qpinidn in the| group wis indiff^ted by one- person's view that the" 
Office of Education should havevno interest-in an agency 's.6valua- 
tion of the validity and ftliabilit^y' of its standards 5nd another 
person's view that all the other ^Tite^ia coul(^be abandoned if 
that one were accurately applied, , ^ 

^ . ■ , ^ ' '■ 

.An agency's inttgrity, ^its evaluation of Bducatlonal outcomes, 
and thf balance between generality an^T'pec if icity in the 
criteria were the three themes that dominated the discussions. One 
of- the small groups gave aal its attention to issues of integrity. 



haying decided that every group into Which the criteria wire 
orqanized was related in/some way to integrity. They also agritd, 
as did^the group at aarge, that integrity is impdrtant in. deter- 
mining educatioh^al quality, that without demonstrablt integrity ah 
agency can have aittla credibility as an evaluator of. educational 

^ ^' !• i: ^ ■ ^ ' ' ^ ' ■ ■ ' ' 

quality. The general impprtance given by the conference partici^ 
pants to integrity feflects the frequency with which attributes 
related to in teg ri ty^-lmpartiality, ' f airhess ^ "and ^independepc of 
judgment"were qonsidereds the influential determinants in the 

: •/ - : vv^' ■ • V ^■ ■ ^ , ■' ^ 

simulated recognition deeisions* ' 



Integrity 



An importint element seen to cnntribute to an agency's in 




rity ^was -thf independence or aUtpnomjUo f ffs decision-making or 
governing body. Ties to other. organizations, or to the institutions' 
or programs accredited. should not introduce the possibility of 
conflicts of' interest or interfere with the fair 'application of the 
agency's standards, Public representation on the governing body 
can help assure autonomy and independence of judgmeht by providing 
that body with an independent viewpoint* Pufellc representation 
aisQ gives a voice to consumer interests that might not.otherwlse 
be heard, The.con ferehce participants thus gave greater importance 
to public representation than did the total group or the agency^^ 
directors in responding to-the questionnaires on importance,^ 
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whether the protection of consumer or public intereets falls 

properly within OE'a* concern for the reri^ble determination of 

; ' \ ' 

f durational quarity was disputed by some,^ wHo held that public 

interests were protected; by other federal igencies, OE , in 

theixTvleWp should not enter that field* Thus the dtgree of 

participint ^reement shifted as issues were defined more, narrowly, 

illusbrarini. the 4mportBnce of the spBCifici,ty with which criteria 

are stated. Integrity in a broid. sense was widely accepted aa of 

high Importance' and pertinent to the determination, of educational^ 

qualdty. But if integj?ity is construed to include the autonomy of 

the decision-making body, some consensus is lost. And if autonomy 

requires the prssence of public representatives on the a||pcy'3 

.governing bodyp the level of agreement shrinks' further* 

* . • ■ 

. Ethical practice by the agency and the attM^on \f the agency: 
to the ethical practices of its accredited institutions or progjtams 
is related to integrity but iB not synonymous with it. jSome 
participants held that.. agencies should closely monitpr the ethical 
practices of their ^hstltutions or programs in .^rea^ such as 
advert Ising p fee structuresi the conteht of bulle 
publ ications p course offerings , and grading practices pT^nd tjiat all 
those .practices directly affec^t the quality of educatiion provided 
and are therefore legitimate concerns of OE, Others:werel equally 

■ " ^ ' ■ ' Ni- / 

strong in their contention that many areas of potentill 'Unethical 
pr act ice-p howevtr reprehensible those practices may ;p^,- are not 
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^Lins and other 
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rtlated to ^edubi t ional quality and are therefore not within OE-s 
irea of conoern. As with integrity in t broader sense, they 
belisved that other agencies, or the institutidni themselves^ 
should monitor the ethical behavior of institutions or programs. 

''Several reasons were given for the importance of integrity in 
thf rBcognition of accrediting agencies* Issues related to 
integrity were^described as generally more poorly defined than 
ot he r" accredit ing issues, such as standards or procedures. 
Closer attention issues of integrity is'therefore required. 
Integrity affects public confidence in the accrediting process and 
the credibility of accrediting decisions* Finally^ because insti- 
tutions or progr'ams are observed only after fairly long intervals, 
their integrity must be relied on to assure continuing compliance, 
with, accredit ing requirements.^ ' ' _ 

Other topici related" to integrity were discussed . One, on 
which "competing viev^ were strongly held, involves the availability 
to the public of. visiting team reports. Some participants believed 
that^ would often prevent critical rffSt^rial fro^ being inclucjed "in 
the reports. .Others believed that the content of the repojts would 
not be affected. One participant suggested that those two contra- 
dictory beliefs be tested empirically. Compromise positions 
between those two were also^ suggested such as permitting ah insti- 
!tutlon or program to withhold release of the repori 4^-^^ii'desirts . 
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Release to' the public of visltir^ ttam rtports Is on,e^ form vof 
public discloaure thaL some participants thought would increase 
public confidence in acDrediting and vyould it the same t ime 'educate 
the pufil ic as to what they can expect of accreditation, ^lother 
^QV^ of public disclosure J which some thotght should be— required ^ 
and which. is not as threatening as the release of reports, is 
complete disclosure of agency policiesi purposes, and procedures, 

y A final point made in response to a question from one of-the . 

""par ticipehts was that the concentration of one group entirely on - 
integrity didHnot imply a belief that . it wa^ generally laokingV-^Qn^ 
the contraryi existing agencies wgre believed to possess it to a 
high degree,' But the public is concerned about issues of ethics 

s.and ^ in teg rityi and the potential for abuse of public confidence in 
higher education exists, 

. ' ■ . . ' ' ' '■ ' : 

P^erf ormance of Gractuates. ^ , 

Monitoring the performafrica 0^ graduates of the programs or 
institutions accredited^ was a second major issue in;the conference. 
beiolte Criterion b6, which cairs qh jgencies to retire "on-going 
pro^ramCs) of evaluation of outputs,-' some participants .were 
concerned that present recognition criteria seem ;ndt to requirs/^^ 
that kind of monitoring; by t^e agenciis and do not require^tftat " 
Institutions* or progMms"' self-studies provide evidence^f student ■ 
learning. Others held that neither DE nor the accrediting agencies 




mvm/Br\!titlB6 to look into the educatianal products of institutions 
, uy/prog rams , In their view, af:craditing agencies should require 
/that effective educational procedures appropriab| to the institu- 
tion's or programj-s purposes be' followed, and tha\ efforts toward 
imprdvement be\ undertaken • ■ But neither OE nor-^accreditinq aqenciea 

, are regulating\ bodies , and instibjtio^ps or prograffis must be left 

' ■ " ' " \ ■ ■ ^"^'"^ ^ " * 

free to choose their own procedure^'' as long as they are not inimi-^ 



cal to student 



learning. J-hus' ''assessment ,of the educational 



product, and th^ compa^i'sons of educatiohal quality that'would.^ 
follow from that , >fBre cohtrasted with the view that the educ:a- 
tional prdcesa ^arlone should be exanined* / ■ \ 

^ An^^l^erm^jate ^position^ was taken by some participants who 
WQ>mi like \tp\s#e /general requirements established concerniihg 
?^- atteh£ion to -^edgc/ationaL outcomes , with the specifics as to the 

- . ' ^ 7 ^ ■■ . ^ ^ ^ " ' . ■ \] 

ways they might be met l\ft flexible. . An agency petitioning for 
retognition mighy be asked \^ describe policies and procedures that 
call for atcreqited instsltut ions or programs to demonstrate. some 
kind of activity directed towird the eicam'ination of their educa- 
.^tipnal products. Sires the er^tire purpose of thft. recognition 
process is to determine the effectiveness with which educational 
quaillty is Judged , thla absence of afWrii;erion related to the 
examiniition of tha educational prpdunt seems hard to defend. 
Again, agreement can be reached on general\ issues; tHe^;sprjcifics of 
their implementation cause trouble* ■ ■ I 



' The! third major issue^ was whether the Qt criteria should bfe 
stated as general requirements or should be spelled out- in consid- 
erable detaia . Both v^ewi had strong { adherents , but thr issue 
dtself was hard to be specific about, and some of the diiagreement 

probably, coulo h?-ive been resolved in particular Instances or 

- . ■ ' > •' ■ ■ 

through, attention to the level of specificity desired. Greater^ 
general it y in thd statement of _the criteria g ives greater flexi- . 
bility to the agencies in the ways they meet the .criteria. It 

also,' though,' alltws greater freedom of interpretation to OE,^ which 

- . j.- ■ '■ 

may lead to disputes with the agencies* Those calling for greater 

specificity, or at least no retreat from the present level of^ 

specif icity, - were concetned that important considerations,!' such as 

those related to consumer protection or/ etHlcal practice, might get 

lost in criteria that were too broadly stated, An^ in'terniedlate 

vi^ew that many participants found acceptable was a /two-level 

approach. The criteria themselves would.be broadly stated require- 

^ments,,' A supplenientary handbook or guidebook would ^then provide 

specific and varied ways the agencies could meet the general. 

cr iteria. ;^ . ' ' ^ ; 

Probably few of the participants would advocate an ex^treme 
vieW^n either side of /the gener^ity-specificity issue, The 
concern was usually over too much or too little specificity. 
One grqup, for example, reported, "Overspecif^icity aisturbed 



everyone,'" and anothir: reported, --Almoet unanimous concern fqr 
overspecificlty Yet,|neither group/ fivored very general criteriaf 
how- mucli speqi ficity was vdesirable /Was the real issue, ^ 

; W^^agrpement was eKpressed on the. importanci.^of the level of 
ipecirioiky in OE ' s, criteria,/ Wide igreement was also found with 
respect to some broadLy stated criteriai such ss the , integrity of 
the agency. But if specificity is increasedi opinions will riiffer 
as to where that Increase in specificity should stop and also as to 
.the kihds'of specification added. Agreement on the value of 
integrity, for ex ample ytnay not be reduced much by including within 
its domain a requir^ent that conflicts of inEirest in the go\/"ern- 

in^g body 'be avoided ./( Greater Feduction in agreement would probably 

_ ^ - - - / ■ ^ ' ^ - ■ ^ . . 

result if^he c ir^oumstances that were to be interpreted as con- 

flicts of lnteres^.,were spelled out #^ Having members of an agamy's 

d^ecision-makipg body serve, as consultants to^prog'rams seeking 

acc red i t a t ion , for exampiei was. seen by some participgnts, to 

const itute a . confl let 6 f interest , But only about one ^person in 

five among those^'who completed the questionnaire simulating recog- 

■ / ■ ■ \ ^ ■ - - ■ ■ ■ ■ ■ ■ ■ ' ' ■ 

ni tion /dec isio6a considered that charpcterlstic to be amof^ the 
five /that most influenced iheir judgments, Jimilsrly, while the 
avoidance of conflicts of interest would be accepted by many as an 
appropriate aspect of integrity /tor OE's attention, 'ethical prac- 
tlces in advertising would attract less support, 

* ' ' A major problem .with any set of crlterii, and one that will no 
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doubt ilways exist, is finding an /appropriate and acceptable degree 

- -of specifi^ifcy* in their formulation. This is a prdblem the present 

project add^issed^ by organizing elements of the existing criteria 
. in^to hierarchical groups of increasing breodth or generali^ty. But 
\ no listing solution can be expected^, The changing contpxt within 
* which accrediting operates was mentioned'by several partiGipants as 
one reason why criteria can always 'be expected to shift' some 

- jextent* The frequehcy md extent of those shifts should d 

^ ' ' ^ " ^ - '" ' ■ . ' ''[^' 

as the focus of attention is moved to higher levels o f generality • 

^ ' ' / ' ■ ■ ' " . ^ - . \ ,: ' . : . ■ -1 ■ 

The conference participants, in focusdng on issuea of impar- 

tiality, fairness, and other aspects: of integrity^ reinforced the 

- .- - . ■ " ■ ' ■ ' ■ A ' 

importance those issues were given in the, simulated^ recognition 

decisions. Virtually all the agency char/acteristicf most fre- 

. quentLy ide^ntified as ar^ng thoSe inflyentiaL in the decisions were' 

. -■ ■ ' ' ■ . . ■ ^ ■ - 

'related to integrity *^ adequacy staff , and effectivadesa of the 
.evaluation proceduresp \ All these are among the attributes/agency^ 
directdfs considered important but dif ficult to^' observe. A majoi 
\ ste^p in, the improvefrient of the recognition process 'would there fofei 
be the development pf more refined prdtedurea for observjing age 
cies* tff^tffivenfeas^. Structural or organizational ,requirement^7do 
not preaint much of a problemi criteria of effectiveness do. 



er|c ^^^^^ 



the relationship of»the OE criteria and procedures to accred- 
o iting agencies is similar to the relabionship o f the agencies 
to' educational pfogrimi md institutions. ■ Agencies do not/speGify 



'1: 

I'- ' 



■the procedures an institution must follow, ilthcsugh specialized 
^genciis may impost sonie curricular, requirements, and they do net 
specify the educational results that must be^ achieved^ They do, 
how^^r, require ev Idence^that the educational program foriows 
qener^llX sound and accepted pnctices. Similarly, DE does not 
impose on agencies any requirer?ents as to the educational products 
they must prorfitite. Yet some set of criteria and procedures to 
assure that the quality of the educational prpduct/is the ultimite 
concern vis necessary . This dilemma between the assurance of 
quality and resppcl for the autonomy and integrity of the agencies, 
is a major problem, for OE as it:is for the agencies in their 
relat ions ^ with programs, ■ 

■ ' ^ i ^ . . ^ . 

The most direct ev Idence . d f the validity of the recognition 
process was in the jydgments of a fairly lirge number of knowledge- 
able people, with different kinds of accrffditing experience, who 
were given tHS critical ^information o)^ which earlier recognition 
decisions had been made. The Ir j udgm^ts -were closely related^ 
,ttr^h& original decisions. The differences were in the direction 
of greFFeF^lenienqy -in the, panel q^/^xperts than 

in the actual decisions * The differences 'also were. greTtest for 

_ " ■ \ ■ ^ * . 

the intermediate dec isions^ involving limited recognition rather 

' =• ' ' ■ _ _ * 

than Qutright denial or recognition for the full = four years* Thus 



the actual decisions were cltarly supported by^ an independeny panel 
0 f experts . ^ ' a- \ j 

' A largier group judges, including almost 100 ancrediting 
agency difectorsj agreed reni^rkably well on the Gharacteristics of 
accrediting agen;cies that ari most important to the evaluation of 
edtjcational quality* Althpugh all the criteria^are related to 
qualities Judged important , the ones judged most important we're 
those that, call for sound evaluative procedures , fair application 
dF standards^ and due ' process . "These qualities were also those 
that most clearly discriminated between agenciee recognized for 
three or four'ysari and those denied recognition or recognized for 
a more limited period* The e^cception was the criterion related to 
public representitiori" on an agency *s governing . body, which t was 

\ ^ ^ ^ ^ ' ■ . . \'-^ 

influential in the decisions^but was considered less important than 

■. ■ ... ■■ : ■ \ ^ ■ • ^ : . ^ " " 

fiiost of the other criteria' by the expert judges'. The qualities 

■-■ ' ■ ' , ■ ■ . — ■ . ! ■ ■ 

most strongly riflected in the recognition tiecisior^ ir'e,^ therefore 

the . qual it ies widely believec^ to be most important in the evalua- 

tion o'f educational quality. 

The ponfeiranci participants restored some importance to public 
T'epresentat ion on an agenrcy Vs governing body by relating it to. 
integrity, which thiy considered possibly the single most iniportant* 
-gpner^at-agency-quai-tty^^^^ content of agerpcy iriteg- 



rity, p-ublic reprasenEation gains importance* The value of atten- 
tion to the relationships among the criteria and theiT components 
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was 'thus demonstrate.d, The.conte^ct of related criterii in which 
any given igency attribute is interpreted iffects its meaning and' 
,itsy import^ce. - 

The conference participanta affirm^ that^ view in another way 
by pqinting out. the importance of stating- criteria at an appropri^ 
ate level of generality—neither overly broad nor too. specific, A 
criiLirion too broadly stated allows toa muct^- variability, in Inter- 
pretation,^ Wordings that is too specific may\leave unstated some 
important related ^aspects of ^ency performance. ,^ Making eKplicit 
\ the jrelat ionships ambng criteria .will "reveal their^road#r ifnpli- 
cations, Just as the components of a criterion draw^^me meaning 
from; their relationship to It, ; ^ 

Despite the influential role played in the. recognition"^ 
pjocess by evaluative procedures, fairness, responsiveness ^ and due 
process, most 6f the criteria were represented in the actual 
decisions* ^ No limited few dominated. Those that .did not discrim- 
inate among agencies g iv^en different recognition decisions were 
criteria virtually uniformly met by. all the' agencies that had 
petitioned fqr recognition. While' these cTiteria were important,, 
the absence of agenciis that failed to meet them kept them from' 
being discriminators. 



The accrediting agency directors were, ssked to evaluate a 
number of suggtfstlons fof revising the criteria. • Those results are 
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discussed In Chapter . IX , None &f the suggested ad^itiohs to the. 
criteria received any ippreciable imount of support , ^ That fact 
and the wide, representation of. the present criteria in the deci-^ 
sions ^indicate that the c'urrent recognition/^rocess adequately 
reflects the appropciate qualities of accrediting agenciis* 

Three sources of evidgnce— the simulated recognition decisions 

by a panel of experts, the. greater rolerplayed 4n the recognition 

decisions by the more' impcirtint agency qualities, and; the represen- 

tation in the recognition decisions of almost all sspects of, the 

cr i te r i a--prov ide support vfdr in f erences that ageord . grea ter 

effectiveness in eval,uati:ng educational quality ^ to recognized 

agenGies' than to/ agencies denied recognition. While some important^^ 

quaUties are less well defined and therefore probably l^s^w^l * 

evaluated trfan others, and whtli ' some may be more influehtial. in 
' " . ■ ^ ■ ^ ^ . ^ ^ ^ ^r^" ■ 

the decisions than ■ Ts justi fled by their importance^ a substantial 

degree'^ validity can clearly be attached tb inferences drawn fjrom 

the rtcoqriition process. 
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IX. AGENCY VIEWS OF THE "RECOGNITION PROCESS 

IMPACT OFTHE RECOGNrTION PROCESS ■ ' ' 

Thp exeeutive directors of 33. accred it ing agenciea visited 

^in March 197'8 were aeked during interv iewe to describe ohangae 
'. ^ t-- ■- ^ ~ 

xv\ the functioning of their agencies during the pist ten years. 

Jhelr descriptive comments were sent to the directors of -the 129 

identified aecrediting and state approval agencies = with' a request 

t'p check which of those changes their agencies had experienced, and 

to stite whether the recogQition process had influenced the change* 

Td assess the influence of the two major revisions of the recog- 

niHon cr.iteria in 1969 and 1974^ thp directors were also asked to 

indicate roughly %he dates of the changes. Completed questiorv- 

naires were returned by 91 agency directors* 71 .percent of the 

total surveyed, ^ 4he compltte results are presented in Appendix 0. 

"Of 43 descriptive statements of changes drawn from the 
initial interviews of agenDy heads> 8 welre related to standards 
( formulation I effect on institutions and ptogramSi and changes in , 
content of emphases)| 3 , with communlcition between the agency and 
its constituehc*ies I 4 with organizational shifts i 10 with accr^ed- 
iting procedur,esi 10 with ihe way the agency responds to its 

: • ■ .. : . '207 . . .; \ ■ 
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constituencies; a,nd 8 with growth or development in the agency's 
capabilities. Table 22 lists the 10 most frequently reported 
changes p the percentages of agencies in which they had occurred, 
and the percentages of agency directors who saw the Office of 
Education as the origin of the changes. 

The two most common changes involved evaluative procedures-- 
improved site visits and self-studies. Almost every agency had 
improved or clarified it s sel f-study procedures , and nine out of 
ten had done the same with"^ their site visits. With these changes, 
the evaluative' function of the agencies wis said to have become 
clearer. Other procedural changes, reported by at least two-thirds 
of the directors who responded ^ made procedures in general more^ 
systemat ic , strengthened the appeals processi increased consulta- 
tion with instituti^qns or programs j and instituted a continuing 
review of substantive changes in programs • The directors believed 
that these irfiproved procedures had increased the credibilit}^ 
of the agencies. 

Almpst as frequently reported as the improvement in self- 
studies and site visits were changes -that clarified and strength- 
ened the agencies- standards and gave more systematic attention to 
their . validity and reliability. Four oth^r changea^n standards 
that had been mentioned by the interviewed agency direcN^^^were 
not widely confirmed. Of those who responded^ only . about -one 
director in four stated that accrediting had become more" heavy- 
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' TABLE 22 



Changes in Accrediting and Approval Practices 

Perctntages reportirg 

that the change,., " Total 

f-hange Occurred Due to OE OE Effect' 

29. Procedures for the self- 
study were improved or * 
clarified. , 98 ^ 33 32 

28* ' Procedures for on-site ^ 
visits were improved.^ - 89 24 21 

4* The agency became more 
systematic in its concern 

for the validity and relit- - 
bility of its standards. 87 30 ^ 26 . 

1. Standards became better 

defined, more e^^plictt. 86 20 17 

27, The- agency -s procedures = - 
became more systematic or 

standardized, SS 21 18 

6. The agency's standards 

were strengthened* 82 24 20 

37, The agencj'- has grown. 79 \7 6 

16. Procedures for dealing with 
complaints against institu- 
tions were improved. '77 54 ^ 42 

11* Information sharing with 

other organizatidns through _ 
confer'^nces J hearings ^ or 

workshops was increased, 77 33 25 

30. An appeils procedure was 

instituted or improved* 74 46 34 

1 I^®^total £fTM^^^ ^racpgnit ion. process is.jthe_^pr^^ 

of the first two coiumns, or the percentaga of agency directors 
who reported that the chanqt had occurred and that it was a 
consequenct of OE's recognition process* , 
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handed or legaMstic. The same proportion thought Iccriditing^ 
now emphasized the identification of weaknesses rather than the 
encQuragernent of improvement, and reported that administrators of 
institutions or programs were more critical oT thp ways in which 
accrediting ^standardi were beir^ defined and applied. Neverthe- 
less, 'o'ne in four of the directors did say that those negative 
changes had occurred, and one director in ten thought standards 
were so narrowly defined that important educationar issues were 
being missed. 

In short, the agency directors were in strong agreement that 
standards from 1969-1978 had become better defined, strengthened, 
and more systematically administered without becoming overly 
restric^rtve or punitive. small minority had som^e reservations 
about other changes. ^ . 

Most of the directors reporting these changes said they had 
occurred gradually rather than within anyTtfe^ned period of 
time* Those who placed the changes in a particular time period 
were, most likely to attribute them to the four-year period .since 
1974. A quarter of the agencies, thouiph, as of late 1978, when the 
questionnaire was completed, had been operating for only five years 
or less, so they were not likely to report changes before 1974. In 
fact, only one of the 43 changes was reported as having occurred 
more frequently in , the 1969-1974 period than in the 1975-1978 
piriod V That c h a ng e , h o w e v eTr wKick^^ 

tition a role in the determination of eligibility for federal funds 

222 
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,had in effect made accreditation mandatory rither thin voluntary, 
waa affirmed by only a third of the directors* 

Responsiveness was a neW rubric for a group of criteria in the 
.1974 revisions, a fact reflected in the frequent changes reported 
in that area after 1974, The most= frequently reported change 
in this group was improvement of procedures for dealing with 
complaints against institutions* This change, and an increase in 
public representation, both occurred largely after 1974, although 
a third of the directors said a general responsiveness to public 
concerns, had been increasing continually since. 1969/ Before 
1974, the specific inclusion of public representatives in igency 
activities had changad only slightly. Improved procedures foK 
dealing with complaints and increased public represintat ion 
reflected new criteria in the. 1974 revisions, and both were said by 
i^majority of those reporting the changes to have been due to OE's 
actions * ' . 

Three additional changes in responsiveness were fairly common* 
More than half of the directors reported greater attention to non» 
traditional programs, ethical practices, and the protection of the 
public against poor educational quality. Opinion was almost evenly'* 
divided as to whether these changes had been continuous or had 
occurred, since 1974, A minority cited OE as their impetus, ' 

The substantive changes in accrediting most frequtntly attri- 
buted to the recognition process were the increase in public 



representation vin the agencies' activities, strengthening of the 
agencies at a result of the self-examination associated with 
petitioning for recognition, and improvement in dealing with 
complaints aqainst institutions. Each of these changes was^said to 
have occurred as a result of the recognition process by from 40 to 
50 percent of the directors (Table 23). More than half of^,the 
agency directors, however, said the burden imposed by OE's recogni- 
tign criteria had increased since 1969, with almost all of them 
understandably attributing that change to QE. Improvements in the 
self-study and appeal procedures, and better commynicat ion with 
federal and state agencies, were attributed to OE by a third of the 
directors. - , 

In general, therif the agency directors reported an extensive 
group of changes since 1974 on top of a continuing pattern of 
growth and change* Most of those changes were seen as related to - 
forces independent of the recognition process , but a third to a 
half of the directors attributed a small groi^ of changes Vdirectly 
to OE and the recognition process. The most widely Teppr ted 
changes involved improvements in evaluative procedures , strength- 
ening of standards, and responsiveness to concerns of the general 
pub lib and other constituencies of the accrediting process. While 
the recognition process undeniably created additionil burdens for 
accrediting igencies, the prevailing view was that the results were - 
^luiBleT 



213 



TABLE 23 

Changes Frequently Attributed to OE 



Change 



Percentages reporting, 
that the change*.* 



Total 



Occurred 



Due to OE' DE Effect^ 



14. 



18. 



39* 



16, 



17. 



10, 



30. 
29, 



The burden impoaed by OE 
requirements increased . 58 

Public representation was 
increased. ' ^ 71 

The 9elf-scrutiny fiquired 
in applying for recognition 
strengthened the agency, 71 

Procedures for dealing with 
complalnti tqainst institu- 
. tions were improved . 77 

The burden of dealing with 
reactions by institutionsp 
progremsy or consumers to 
OE 's requirements ihcreased. 47 

The igency became aware of a 
need for better communication 
with federal and state 
agencies, ' .67 



.An appeals procedure was 
ihstituted or improved. 



74 



Procedures for the self-study 
were improved or clarified. 98 



93 

70 

64 

i 

54 



77 



52 
46 
1^3 



54 



50 



45 



42 



36 



35 
34 

33 



The total (B f feet of the OE recognition process is thi product 
of the first two columns i or the percentage of igency directors 
who reported that the change had occurred and that it wis i 
consequ enee o f OE ' s recognitlon_pJOcess _ _ 
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SUGGESTIONS FOR IMPROVING THE RECOGNITION PROCESS 



In February ISTS, shortly before the visits to a sample of 
accrediting agencies, early versions, of two questionnairt^^^^e 
sent to their directors for review and comment* One was %he 
questionnaire, composed of statements from the DEAE staff reports, 
that asked for judgments on the importance of various accrediting 
agency characteristics. The other, containing the same statements, 
asked for judgments about the ease or difficulty of observing those 
Characteristic^* ^ 

Twenty-five of the sample of 33 accrediting and state approval 
agencies had been selected randomly to represent medical aad 
nonmedical agencies, and regional, state, and national agencies* 
The national agencies were furthar grouped for s^ampling intp those 
dealing with programs that were dominantly pre-baccalaureate 5 
baccalaureate,'^ or graduate. Eight additional agehcles were in- 
cluded because they represented particular points of view or large 
numbers of programs* The directors of these agencies .were asked to 
review the two questionnaires and to comment on them and other 
aspects of the study in interviews to be conducted the following 
month. Staff members from four regional offices of Educational' 
Testing Service visited each of the selected agencies and reviewed 
the form and content' of the questionnaires with the director. The 
di-r-ae^tor-^^were^aisQ-^asked^to=-Comment.-.o , 
tion process on their own agencies and to^ suggest changes in the 
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process* The suggested changes were later sent to all accrediting 
agency directors with a request that they indicate their agreement 
or d isag rtement • Tha questionnaire and response distributions o f 
the 94 directors who responded , 73 percent of the total of 129^ are 
presented ifi Appendix E, * ^ ^ 

The-^ 55 suggested changes were grouped into mix categories* 

/ ^ . 

Ten suggestions asked for additional information about the recag- 

nition process from DEAE. Twelve were comments about perceived 

i ■ 

inequities or inconsistencies in the recognition process that 
implied a need for change. An example was the comment,/ with which 
about a^ third of the directors agreed, that DEAE^s procedures for 
due process were inadequate * Another twelve suggestions concerned, 
modifications in DEAE's role or function. Six suggested alterna- 
tive ways the recognition . process might be structuredv as, for 
example^ by r emov ing any concern for educational quality from, 
the determination of eligibility for federal funds and. leaving 
finanrial responsibility and accountability as the only considers- 
tions. Finally^ nine suggestions recommended, additions to the 
present criteri'a, and six recommended deletions, 

. Ciearer Direction from DEAE 

The ten ^ recomm end at ions for additional information and 
guidance from DEAE were strongly endorsed by the directors, with 
from 52 to 82 percent of the directori either strongly agreeing or 
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checkfng the sixth point on the seven-point" scale , whi 
betwten ^^Agrae" and ''Stcongly Agree." The three most frequently 
endorsed recommendat ions— by at least three-FDurths of the direc- 
tors-»were that DEAE pubiish its pMcedures for evaluating ^en- 
cies in de tail , specify the kinds of evidence acceptable for 
meeting each criteriort, and provide agencies with a copy of the 
staff report prepared for the Adv isory Committee * At least 40 
percent of the directors gave r ating s /o f S /or 7 to the other 
recommendations that cilled for additi/onal informstion or DEAE 
guidelines for agencies. These included more explicit definitions 
of some of the .criteria , more detailed information about the 

reasons for t^e recognition decisions, and the provision of more 

■ - ■ ■ ■ " ^ ■■ ./ ' ^ ^ ' ■ . ^ 

information about the recognition Process through publicatiGn 

. . ^. ■ ' 

of th6 responsibilities of DEAE arid the Advisory Committee^ the 

criteria and procedures for selecting Advisory Committee .members^ 

and the effects on an agency of different recognition decisions; 

These and the suggestions in othfer groups for which the difference 

between the ratings of 6 or > and the ratings of T and 2 were 

greater than 50 percent^'are shown in Table 24, 

■ ^ The responses t£^ thep^ recommendations strongly: indicated 
a need for additional guidance to institutions and programs as 
well . as : for n|ore detaUid information about DEAE procedures for 
evaluating agencies. The keen interest shown by agency directors. 
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fept that even^ t'ht"' least strongly supported stittmint^ irT-this 
aertlon--*calling for greater clarity in the wording, of the recogni- 
tion criter ia--wps given a rpting of 6 or 7 by 42 percent of ^he 
directors* That| and the other two .recommendations in this group 
that were least strongly endorsed--for improved directions for 
preparing petitions^ and for a timetable for the submission of 
support ing materials--constituted clear exmples of the kind of 
additiohal information agencies wanted from DEAE» This concern of 
th^ agency directors had been anticipited by DEAE* A guidebook^ 
which constitutes Part III of this^report, specifies the kinds of 
..evidence petitioning agencies might present to DEAE to satisfy each 
criterion for recognition as well as procedures" for petitioning for 
recognition* 

TABLE 24 

Suggestions Receiving Strong Support 

^ / Suggestion Percentages of two highest 

ratings minus two lov.^est 

3* DEAE should provide agencies with a 
copy of the review summary presented 
: by its staff 4to the Advisory Committee* 82 

8* DEAE' should specify the kinds of . r 

evidence it considers acceptable tor 
meeting each criterion. .77 

1* The Division of Eligibility and Agency . 
^ EvalMatipn (DEAE) of the U*S* Office 

of Education (OE) should publish in . 
-^.jdat^Al ^iti procedures for the . 

evaluation of agencies', ""\ j^^^j^ 
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TABLE 24 
(cont'd) 

Suggestions Receiving Strong Support 

Suggestion Percentages of two highest 

ritings minus two lowest 



31. Institutions receiving federal fun^ds 
should be held legally accountable for 
the honesty and accuracy of the infor- 
mation they publish and disseminate 
^ about their educational pilgrims and 
financial status. 67 

5* The responsibilities of DEAE and of 
the OE Advisory Committee should be 
published* 65 



4* Procedures and, cri*teria for selection 
of OE Advisory Committee members should 
be published* ■ ^ 

20. Ar accrediting agency staff member should 
. be. present at the time of the agency's 

review by. the Advisory Committee* 60 



9*. The commissioner's report should provide 
more detailed feedback on the reasons for 
the recognition decision. 56 

25*. DEAE should hold open meetings to give 

agencies an opportunity to provide input on 

policy changes* 55 

17* Hearing procedures for agencies should 
" be structured to ensure that appropriate 
issues are covered. 55 

7. A timetable for submitting materials to 

DEAE 'ghould be provided to agencies. - 5J 

2. OEAE should provide more information on 
how an agency will be affected by a 
particular recognition decision^ ■ 51 



S 
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Inequ*itlei or Inconsistencies in Recognition 

The most frequently end ^^ed commen^ on inequities or incon- 
sistencies, with a half or mDre of the directore Indicating strong* 
agreeifient (ratings of 6 or 7) and no more than three or four 
indicating strong disagreement (ratings of 1 or 2)^ were the 
following (paraphrased)? 



20. An agency staff member should be present during the 
Advisory Committee review. 

17* Hearing procedures should ensure attention to appro- 
priate issues* 

19, The. Advisory Committee should read the petitions as 
well as the DEAE staff reports* 

14* DEAE should meet the same criteria the agencies meet* 



nThe first three recommendations indicated a fear that un- 
favorable decisions may be reached on the basis of Incomplete or 
erroneous information. The nped for specific procedures to prevent 
and redre'ss such decisions was implied* The fourth is a broader 
.concerrij which suggests that DEAE be concerned with issues of 
responsiveness , revaluation, standardii due process, integrity, and 
orgfinlzational structure to the same extent thpt it places those 
regyirem^fnts on agencies. The fear thit bad decisions may^ result 
from bad information can be reduced in several ways other than 
by the specific suggestions cited abovi* At presenti for example, 
agency staff members do have an opportunity, to present' information 
directly to the Adviiory Committee: and mdy be present during the 
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Committee's deliberations. The issue in general ie a defensible 
one, howevery, and' deserves attention in any restructuring of DEAE's 
procedures. - 

Most of the other comments in this group described issues 
^bout wtiich la.ge proportions of the directors had no opinion* Yet 
, appreciable minorities, on the order of 20 to 30 percent, were 
critN^cal of the recognition process in virious ways, feeling, for 
example, that large, povyerful agencies got preferential treatment 
(number 12), that the interpretation of the criteria was incon- 
sistent from year to l/ear (number 13), and that DEAE's procedures 
to assure due process were inadequate (number 15). 

DEAE Functions ' ^ ' 

Aiinqng the suggestions related to the role or functiona of the 
Office of Educatipn, the^^two endorsed far mpre strongly than any 
others were, first, that institutions receiving federal funds be 
hel^ legally accountable " for the accuracy of the= information they 
provide about their educational programs and financial status, and 
second, that DEAE hold open hearings on policy changes* Two other 
suggeations also received strong support. > One was that' a vafiety 
of information about pirticular institutions and programs—changes 
in el Ig ib il it y , ac t ions of the FTC and the CDurts, and accredita- 
tion or approval actions— be more widely published. The other was 
that- the' extent to which the practices of Jprograms ijrid insti^^^ 

■, 1 .■ ■■■ ■ , . ' ■ ■ ■ 

. f * . •■ '■ . ■ • ■ 
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were ethical shoyld be a concern of the Office of Education in 
determining eligibility for federal funds. ^ 

Some other euggistiona in ihii group are notable because of 

• the absence of any clear cdniensui* For eximplei the use of 

_ ...^ ........ % 

racognltion c^dteria to promote experimentation received only 
lukewarni support. And the recognition of more than one agency 
operating in a particular educational area was viewed more nega- 
tively than positively, but not by a great margin. 

X Restructuring the RecdqnitiQn Procefrs • 

Only, one of the six sugges^tions for restructuring the recog- 
V nitipn process reuieived much support. It c alle fj for the establish- 
ment of an Appeal body to which agencies denied recognition could 
.^have recourse. About a third of. the directors gave this suggestion, 
judgments^ o\f 6 or 7, while 7 percent ptrongly rejected it. A 
j suggas,t^ipn that ^eligibility for federal funds not involve standards' 
v>of educati quality was bverwhelmingly rejected* Only 12 

p6.rcent of the directors gave it even slight endorsement , while 81 
percent r^jjct^d^-irtrr H'wo sLggestions in this groi^ are notable, in 
J the^v.virtually even spread of opinion from one^extreme. to the other. 
Omilrwas that the responsibility for recognition be delegated to an 
- independ^nt-rbody , such as the Council on Postsecondary Accredita- 
-^^tior^S ; The other suggestion'^ai similar i calling for the review of 
ah agency*! operations to be carried out by^an independent body. 
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About 80 percent of t+it directors sxpressod.an opinion, on each of 
.these, end on^each, any sizeble "group that expressed strong 
agreement was matphed by-an equallv silable group, that expressed 
strong disagreement. : ^ 



Changes in the Criteria 

None of \he suggested additions to the criteria wis eYidorsed^ 
. by the directors, and only one suggested deletion recfeived even 

moderate endorsement* About half of the dlrectQrs agreed '^feomewtiat , 
/ and about a quarter agreed strongly, that the criterion calling for 
"•agencies to be concerned with .the affirmative action policies^of 

its programs or institutions be abandoned. (Although the. relevant 

■ . - . . ' s i e 

criterion calls for agghcies'to foster nondiscriminatory practices, 
some directors have- interpreted this to include afflrm^etive 

action as well.) The deletiop; of several other criteria-had' been 

: . ' > ^ ' ^ y , .' ' / ■ ■ ^ . ^ - ■ 

suggested, but none was supported for deletion by more than about a 

third of thF directors, and in every, cast, more dir^ favored 

retention- than deletion. These included the ^criteria related to ^ 

filling an identifiable need, requiring public representatives on 

; the decision-making bodies , having a visiting team member pre^^^ 

' '/^.'^ ■ . ■ ■ ' . , " ■ - ' . ■ ■ " ' ' ' ' V . 

at the evaluatioh of the visiting team report, fostering ethical 

prictlces, and encouraging innovation. 

■ .■ ' ■ ' ' , 

- v: QTfly^one^suggfi^^ ev en slight -support — - 

the estiblishing of special procedures for evaluating off-campus 



v.....,,. 



ERIC 



223 



progriams*^ Although more than s hal f o f the dirtotors eithe4 
r-exprasstd no opinion Qt took a middle view of that Buggestion, ' 
slightly more dtriDtors favored than rejected, it * Certifyihg 
site-visit reports , conducting lurpriie .audits, and gathering 
—specific information about the integrity of chief exeeutlyg, _ 
officars were all itrongly rejreted. 

Responses to the Suggestions 

The main thrust of the directors' recommendations was toward 
a fretr flow of information about the fecognition procesi* Of the 
12 most strongly endorsed suggestions, listed in Table 24, all 
.but one involve the flow of information, pirticularly about the ' 
mechanics of^the recognition process — ^how the information that 
. reached the Advisory Committee influences its decision about an 
agency, ind/the reasons for its decision, ' The one suggestion in 
Tabl^.24 not related to the flow of information called for ipstitu- 
t ions to be held Ifegall y ^accountable for the accuracy of the 
information they publish. ' ^ 

The annually published list of nationally recogniied atered- • 
iting agenciea ind associations includes some of the information 
desired by the directors. The responsibdlities of DEAE and the - ^ 
Advisory Comtnittee, for exampli, sre listed in,it each year. . 



Thus suggestion-- number 5 (Table 24), endorsed by almost all the 
directors, has been met. In other cases though , the dtsirtd 
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infdrmation is skimpy 



Commiss ioner * It cou 



■/ . I ^ 

at best. For example , a ohe-sen£tnct pari- 



graph in the annual , publ icat ion of , recognised agt^ies tells an 
agency where to write for infoEmation about becomi/ng listed by /the 



d as easily, provide briS/f information afaoQt 

the petitioning, evaluation, and decision prqpiss that prectdes. the 

listing . ' 

^ . 1 

As was mentionec^ earrier in this/'chapter, a set of guidtjlines 



has been prepart d/that specify thk kinds of information agencies 



may subnfit as evj/dence of satisj^ction of the. various criteria, and 
the procedurep"^ to b follow^e'd in petitioning for recogn|tion. 
Numbers 8 ah^d 6 among the- suggest ions in Table 24, calling for 



the specification of acceptable kinds of evidenDe and dirictione 
for preparing peti^i^ns, are therefore being met* Other informa- 
tional needs could be met more fully if more detailed descriptions 
of the mechanics of the decision process, the selection of Adyisory 
Committee merfilDers, and other details of recognition were published* 

Some o'f the suggestions in other areas are being net with 
current procedures. Open meetings on policy changes, for f^ample, 
have been held intermittently since DEAE was established in 1968* 
The strong endorsimeh't of this practice, as indicatedi by the 
directorsV response to number 25 among the suggestions, indicates 
that that practice should continue. - 
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SUhMARY ^ 



Thi two kinds of inrormation described iboya-^chinges that 
have occurrSd in accrediting during the paet ten yeiri and changes 
in the recogniition process that agancy diractors would lika--shQW a 
subsiantlai; strengthening of^^^pcrediting: and ;pn 
with rtcognition. fl^ccred'lting has been improved prifnarily through 
more effective use of the saif^study and site: visit in evaluations 
and through greater concern that accrediting be responsive to the 
Interests of its various conVtituencies, including the general 
public. The care and attention given the formulation and main- 
tenance of accreditihg' s.tandirda also has increased* . Most of 
these changes have been seen as natural developments of the 
accrediting process, but OE's criteria foj recognition have been 
given credit by apprapiable numbei^s of accrediting directors^ — from 
a third to a half of a highly representative sample'^*-for the 
ihcre^ased responsiveness arid for som^ of the improvement in 
procedure's* 

Despite sortie occasional sense of irritation at the increased 

* '' ' ' ' . ■ • , _ ' ' ■ 

burden the recognition process imposes on accrediting agencies, 

. ,. ' ' ■ " ' "^-^ , ■ 

their directors were generally satisfied with the criteria *and 

=^ ■ " . ' ■ , ' * 

their application* , No suggestions for substantive changes Jn the 

■ ^ ^' " ■ " ' ^ 

criteria, received .appreciable support^ although several of the 

exis ting criteria Judged to be less important^ than most— those 

concerning public representation on governing boardSj ethical 
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practiceii hiving a member ^o.f the visiting^ ttam present at the 
evaluation of its report, and encouraging innovation—would not be 
missed if thty were abandoned. ^ . 

The most friquently^recommended changes in the recognition 
process called ^'br some form, of Improvement^ ^ the flow of infor- 
mation between DEAE and the agencies* Related to that concern, was 
a desire for procedures that would, minimize the intrus ion o f 
m i B in f 0 rm at ion into recognition decislonsp and for an appeal 
proceps that would counter the effects of erroneous information or. 
flawed decisions.. The general structure and functioning df the 
recognition process have been well' accepted. Adjustment of some of 
its details j however, was judged to be desirable* 
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X. DIFFERENT lAL WEIRHTING OF CRITERIA • 

The published erlterla (ftpptndix A) on which recognition of- 

accrediting agenqlti is baaed are org aniied . into a hierarchical. 

pattern CDnsiatirng^^o f four brQid areai, 19 iubareasi and 24 more 

specirie criteria, some Qf.which are subdivided further to product 

a total of/ft4 criteria^ Yet the^ icope of the various areas, 

subareai, and criteria do not^'^lwayB match their hierarchical 

poiltion* Somt subareas seem broader and more important than iome 
- ^ ^ / ■ ■ ■ ' - ■ ' ' " 

areas, while other subareai are les£ impbrtant' than some of the 

criteria* For example j tht fourth major area involves agency 

autonbmy in farming independent judgmenti and guarding against 

conflicts; of Interest, In contrait', a subarea of another, major 

area includes th| requirements ^that-accredited InBtitutions meet 

established standardSi thit policies and procedures be fairly 

applied, .and Jthat decisions be objective and impartial. .Apart. 

from the apparent overlap in these Jtwo criteria, tha scopt and 

importance of the second^seem/ at leaBt as great as the fir^t, 

which navertheliss stands in a higher position hie rirchlcglXy * 

Similarly, Criterion b? Whlch';^f|ts .for the encouragement of 

f xper imentation and innovation, Is hierarchically above but less 



im port ant than C ri te r io n b 3 v i i i^ which Is ^ c one erne d" with tht 



i/ - ■ ■ '/ -^J^ 



/ 
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establiihrnent of procedures for appeals. 



The recognition decisfonr whicti consiati only of oleiLifying 

an agency, ai having tieen denied recognition or granted it for from 

'one to four yean, it intended to rpflict the; quality of an aecri^- 

iting agency in tirms of all 44 of the criteria, and the broader 

Bubareas and -areas into which they are organized* ' That hierarchy 

■ ■ * ■■ ' ^ , 

ical structure^ however, because it ii not always coniiatent with 

actual differences in the scope and importance of the character- 

^istlcS' inoluded, may not help the Adv4aory Committee reach a 

decision/ In arriving at their recdmmendation to the Commissioner, 

the Committee weighs an agency's 'Weakneises against its strengths, 

ass#sslng^ the seriousness o.f^the deTiclencies and the likelihood 

that^ they will be , corrected within a Teasonable period .of time./ In 

effect, the Committee Implicitly gives greater weight to some 

criteria, probably without regard to their hierarchical position, 

and it may even vary the weight of one criterion, , quite properly ; 

and apprbpr lately/, depending on the agency's per formarice on soma 

0 the r cr i t er ion t Questions about an agency's autonomy, for 

example, may not count as heavily against an agency^with wall- 

d^e fined procedures for guarding against uhfa4Jr or biased decisions ; 

as against another agency deficient in the second area as, well as 

the first. Also, agencifes^ with the same number of deficiencies may 

prbperly ^receive different decisions because of their different 



patterns of deficiencies* 



- , 229.; , - . ^ ; / 

■ ■ : - ■ , ■ ■ \ 

Such e process of informsl md perhaps unexpressed weights 
maXleay j^becaas^ of uneisineea over its lick of systtm or speci- 
ficity^j to efforts to focmal iie and Standardize the process by 
specifying fixed weights to be applied to each criterion. A system 
of nirefulay established weights has Intuitive appial, part^icu- 

larly when rnany considerations, varying in Importance, enter into 

■ . ■ ^ , -■ " ■ ^ ' 

the diciSiGn* The mor©^ important criteria should ,presumiblyi 
carry greater weight* Some justification can bIio be found , 
howeveri for giving greater weight to the more reliably observed 



criteria (Fitzpatrick 4^ Morrison, 1971). As important as 
quality m^ay be, if it cannot be reliably observed, giving it^much 



weight in the- decicion 'seems unwise. The following discussion 

t : . ^ ' ^ ■ ^ ^ ' ^ - ^ ' ^ f 

therefore covers fixed weighting^schemes based on importance, on 

reliabitity, and on both combined. \ / 

■ . " ■ ■ " -■ ■■ . , ' 

CALCULATION OF WEIGHTS ■ 

\ ■ . '■ ^ ■ " ■ ■' ■ ■ 

\ Studies of the- use of weights in decision processes have 



usually been directed toward the Improvement of predictions. In 
hir^ingM ife insurance salesmen, for example, years of prior sales 
eKperienc\, years of postsecondary education, scores on a test of, 
generpr intel^aS^'^W, and scorea or. i personality inventory related 



to salesmanshipNnighit^ alllbe congidered important in se^^ecting 
the most praising applicants* The relative weights of those pre- 



ictors of perforfijance can be determined by finding the^cesmbination- 
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that shAws the highest ^relationship with Ipter salts recprda. In 
r/idat applications of welghDS, .lome buqH acctpted. criterion is 



availaBlfe to permit the calcuMtion of an optxinal set of weighbi* 
. . Th,osB weights are then used in Vuture. selecti&n decieiona on the 
assumption that tht predictive relationships had remained un- 



\ 



changed. ^ ^' 

The recognition process , howbver,\haa no obvious standardV' 
of merit for accrediting agencies* As aKconsequence^s no set 0f 
weights for ihe vario''us recognition criteria can be established 
that wo^l^d be based on an acceptable predicted standard of^ggr- 
formance . While the. relationships of the critefia" to past recog- 
nliLon decisions can be calculated , . they would only i^ndl^ate- 
how the. or Iterla had been Informally weighted in the p|f fc|- not 
What their weights should be. In the absence of ^ tftanyard of 
accreditlng\ agency petformanoe, Illustrative sets a^weights .^te 
developed fr^pm the agency directorsV Judgments o/ import'ance' and 
rellabllltyV 

As repor4ed in Chapter VI| agency directora'and' a variety of 



other knowledgeable persohs judged the Importance of a number of" 
characteristics to an agency's ability to evaluati educational 
;-|d^lity/ Each^haracteriitlc was associated with at least one of 



* the facognitlon' criteria. The average taipbrtance of t'he statements 

' . ' ' . ^ . ^ y i . . ■ ■ = ■ 

— dijLe^atl v associaCed wit h each cr iter ion p rovid ed an o v er al 1 JMdg* 
ment of the Importance of the criterion. Values / of statement^ that 



were not direct derivatives of the published criterion were not 

■included in its average valuer even though they were usually 
pertinent to it* For example. Criterion b8 specifically requires 
that accredited institutioni meet published stindsrds, thit the' 
standirris and prQcedures be^falrl^ ipplied, and that deciaions be 
^reachled impartiilly and dbjectively.^ The importance of ' statementB 
embodying these three elements^f ' the criterian were averaged foP^ 
Iti overall importance^ EKcluded were other statements related to 

. tne criterion but not an explicit part of it, such as, "Ttje agency 
mo^tors continuing compllanda with standards through the sub- 

'.mission of regular reports." The judgments of importance from which 
weights were derived for each, criterion (Tablfe 25) therefore , 

^^refeTred to the cri'teria in a narrow and somewhat literal sense*; 



They did; riot incorporate broader implications of thtf criteria that 
occasionally appeared in the descriptive statements, nor did they 
appay^to the stat^^n^ except v^en a single statement 

happened to be a direct reatatemd^t of the criterion. 

_ In the early planning of the study, variations among different 
groups of Judges in the importance assigned to various criteria 



were expected to lead to sever il seta of weights—different .weights 
for grdups viewing' accrediting /from different perspectives* » The 
high/Tdegree of agreement across the various groups of Judges 
reported in Chapter VI ma\ie that unnecessary* The weights, in Table 
25, for importance as well as observability, were therefore derived 
from the Judgmenti of the agency directors. : — 
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- . TABLE 25 , 
irativt Weighti of Criteria 



b3iii 
b3ii 

i3iii 
bZiiA 
b3vii 

b3vi 

b9 



Criterion 
Written report ia provided-. 

Visiting tasm maeti all 
groupa. 

Self-study is required* 

Standards are publlahed. 

.Adverse decisions may be 
appealed* 

Accreditation is withdrawn 
only for cauae* 

Reeviluation occura at 
reasonable intervals r 



' Weight Weight ^ 

due to' - due to ^ 

importance®^ pbaervabllity 



aJillB Guidance is given, ;for aelf- 
" i itudy and visit0 
' . ' * ■* 
b8 Standarda are fairly applied, 

di'Clalona Impartial* 

aZi - Staff and procedurea are 
adequate. 

b3iv CEO may^^^. rispond to the 
report . 

a3l " Definitiona and prooedurea 
■are c'leir 

bJvillC CEO is given spioirica of 
,. ■ ippeal deciaion. 



9.60 (3 )d 

9.6Q*(3) 
9.60 (3) 
9.50 (3) 

9.50 (3) 

.9.40(3) 

9.40 (3) 

9. 30 (3) 

9.12 (3) 

8.90 (3) 

8.70 (3) 

8.35 13)- 

8.30 (3) 



9.32 (3). 

8.66 (3) 
8,52 (3) 
9.32 (3) 



8.76' (3) 
8.6|, (3) 



6.24 (2) 
6.24 (2) 
9.39' (3) 
8. 83 (3 ) 
9.11 (3) 



Wtight ; 
due 
to both° 

9.46 (3) 

8.79 (3) 
8:65 (3) 
9.36 (3) 



8.71 (3) 
8.57 (3): 



6.02 (2) 
5.87 (2) 
8.64 ("3), 

7.99 (3) • 
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• = TABLE 25 .^-^^^^ 
*^ Coraparative Weights of Criteria 



d1 



Criterion 

Independenoa of Judgmenti 
Is protected. ^ 



b2iii Notlci is given prior- to 
ri vision of standards . 

a2ii Financial resources v 
are adequate . 

dZ * Conflicts of .interest are 

avoided^ .. .. ' 

bJviii Appeal procedurea are 
published* 

falii Pc/rposes are clear* 

b3viii6 Hearing be fort appeal body^ 

: is pruvi^ed, 
< ■ / ' ' V 

>b3viiiA Stptus is not changed 

^^^^ during appial* 

.c2 Standards an^d procedures ' 

are. regularly reviewed. 

b2iv , Procedures for review of 
complaints are written. 

b1i Rights of all groups 

: art considered. 

aiii Scope of actlvltiti is Djear. 

bZiiB Proceduret are pablishedi 



Weight, 
due to 
Importance' 

8.20 (3) 



8.10 (3) 

7.95 (3) 

^.80 (3) 

7,80 (3) 
7.50 (2) 

7.50 (2) 

7.40 (2) 

7.35 (2) 

7.30 (2), 

7.18 (Z) 
7.00 (2) 
7. 00 (2) 



Weight" 
due to, 
pbaervability - 

6.28 (2) 



8.62 (3), 
. 6.90 (2') 
7.99 (2) 

9.32 (3) 
9.11 (3) 
9.11 (J) 

ft 

, 6.24 (2) 
6.84 (2) 

■ 8,72 (3) 
9.39 (3) 
9.32 (3) 



Weight 
due ■ 
to Both^ 



5,44 




7-.38 (3) 
5.80 (2) 
6.59 (3) 

7.39 p.) 

7.22 (3) 

7.13 (3) 

4.85 (2) 

5!28'(2) 

S.62 (3) 
■6.95 *(3) 
6.90 (3) 
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a2v 

b5 
b31 

1)7 

b6 J , 

bZilE 
allli 
bZliC 

b21lD 



b2i\ 



TABLE 25 
(cont'd) 

Compirative Wpighti of Criteria 



Criberian 
Agency is widely icceptad, 



Viiiting ttami ;include. 
a nonstaFf-member* 

Standards are evaluated. 

Initial evaluation Is at 
CEO's request.* 

Innovation la ancQuragea, ^ 

Evali^tion oF*outputs is 
rtquiretl. 

Owntrihlp is puBlished. 

Fees are. reisonabla. 

Eur rent st atus md^ next 
rtview are published. 

Membership of board la 
published.' ;= 

Ethical practices are 
fostered. 

All groups are rfpresented 
on governing bodies. 

Preaccreditation is- relitfeJ 
to accreditation. 

Public is reprtiented 
on the board. 



Weigf^t ^ . Wfeighfc 



due 'to 



due to 



Weight 
due 



importance observability to both 



7.00 (2)^ 

7.00 (2) 
6.70 (2)-. 

6.60 (2) 
'6.40 



"6.30 (2) 

6.20 (2) 

6._^00 (2) 

5.90 (2) 

^.90 (2) 

5,50 (2) 

4.60 (1) 



4.20 (1) 



4.00 (1) 



7.92 (2) 

5.78 (2) 

8.62 (3) 

5,96 (2) 

8.86 (3) 
9.32.(3) 

6.98 (2) 

,9.32 

9.32 (3) 

6.1-4 (2) 

8.13 (2) 



7.92 (2) 



9.22.(3) 



5.86 (2) 

4.09 (1) 

6.01 (2) 
■4.03 (1) 

5.90 (2) 
6.11 (2) 
4,43 (2) 

5:81 (2) 

5.81 (2) 

3.57 (1) 

3.95 (1) 

3.52 (1) 

3.90 (1) 
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TABLE 25 
(cont'd) ^ 

CDmparative Weights of Criterii 



b3v 




Criterion 

earn' member is present at 
eviruation of report. 



Weight 
' due to 



Weight 
due to 



a b 
import ince observability 



3.60 (1) 



9,18 (3) 



Weight 

due 
to both 



3.49 (1) 



Not juc^idi 

a1i Scope is national or regioiiul. 

Perionnel are competent* 



aZiv 
bio 

03 



References to accreditation 
are clear * - ^ 

Agency ha^ two years' Experience 
or mope 



The weight Is the Index/of Importance, described in roothote 2, p, 103, divided 
by 10. ^ / 

b . 

"The weight for observability ^ Wi is based on the median judgment of observa- 
bility, p, according tq the equation, W = 3,5R - 4,4, to place these weights on 
approximately the same scale as the weights given to importance. The equation 
was derived by equating extreme values on the two scales and assuming a linear 
relationship between them, 

"The cqmbined weight is the product or the two individual weights divided by 9*46j 
the square root of the maximum value of the product. 

^Fiqures in parentheses are simplified, single-^dig it weights. 
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Computational PrQcedures for the Weights inTable 25 



For sumniirizir^ the results of ^ almost 100 individual judgments 
of importance for each statement p none of the usual indicators of 
central tendency ^ such as the mean or mediani seemed appropriate. 
A seven-point scale was used for the ratings to provide some 
discrimination amorig the large number of statements expected to be 
judged high in importance while still allowing for some negative 
judgments. The J-shaped distributions of the ratir^s confirmed 
that expectation. The median rating, which would be appropriate 
for more uniform distributionsi does not give adequate considera- 
tion to the comparatively few but important highly negative 
responses, since the large number of positive respdnses produces a 



median value near the top of the scale* The median does not 
discriminate, then, between a response distribution in >^ich all 
the responses occur on the upper half of the scale and one in which 
a small but important number of rtsponsas are at the low end of 
the scale. 

The index adopted to provide a More informative summary of the 
rat'ings for each accrediting .agency attribute was the difference 
between the percentages of responses in the top two and bottom two 
categories o* the seven-point scale (see Footnote 2, p, 103)* This 
has an intuitive aj. eal in combining the more definite responses 
at either end of the scale while dropping out the intermediate 



\ 
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judgments* It can be interpreted as the net percentage of high- 
importance responses* The disadvantage of this inde^ is that it 
does not distinguish between U-shaped and inverted-U distributions , 
although neither does the median nor. the mean. With the present 
data, inverted-U* or humped distributions of any kindj- did not 
occur. 

# 

The ipdex of importance j or the net percentage of judgments 
of high importance attached to a statement or attributt of an 
agencyi ringed- from 6 to 96^ although only '5 of the 80 values were 
lower than 30, while 15 were 90 oi^ higher* When the indexes were 
attached to criteria through ^averaginig the values of the statements 
that' d irectly reflected th© content of a criter ion * the values 
ranged f ro^ 36 to 96, Those ^values wer© arbitrarily divided by 10 
to produce a set of weights .ranging from 3*6 to 9,6 (Table 25), 
retaining'a ratio of 2*5 for the highest to lowest weight j while 
putting ^the weights at an order of fnagnitude most people are 
comfortable wi'th-^-a scale of approximately 1 to 10. 

The judgments of observability were made on a four-point 
scale, since large numbers of extreme ratings were not expected. 
The median values ranged from 1*84\to 3*94 and showed a distribu- 
tion very similar in shape to that of the indexes of importance. 
A linear adjustment could make the two curves almost coincide. The 
median judgments of observability were adjusted to put them on a 
scale similar to that of the index of importance by 'throwing out 
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the three lowest values of the two curves. The result was the 
following equation, in which W is the observability weight on a 
scile similar to the scale of importance and R is the median value 
of abservability i ^ 

- W^s 3r5R — 4^4 - 
The resulting range was 2*04 to 9.39, 

A multiplicative relationship was desired for the composite 
v/elghts^ since an eKtreme score on either of the two component 
scales should not have its effec^ diluted in the composite. If an 
irithmetic mean ^were used, a low weight for either' importance or 
observability would be balanced to a large extent by the other 
weighty and the low weight that should be given to any quality that 
ns either of negligible importsnce or virtually, unobservable would 
not appear. The geometric mean—the nth root of the product of n 
te rms--prov ides a multiplicative relationship but also dilutes the ^ 
effects of low values. A hybrid form of mean was used, therefore, 
consisting of. the product of the two weights divided by the highest 
qeometrlb mean oF any pair of- weights. Thus the composite weights 
are a function of the product of the two component ,weights, but 
dividing by the highest geometric mean puts them on, a scale similar, 
to the other two scales of weights and keeps the composite weight 
low whenever either. of the component weights is low. 



I 



239 



THE APPLICATION "of WEIGHTS 

■■ 

Applicition to recognition decisions of any of the three, sets 
of weights of Table 25 would presumably coniist of assigning an 
agency the nuFnerical value of the sum of the weights of all the 
crite^ria it m tpLal values bilow a pred eterm ined 

level would be denied recognition, and successively higher values 
would be required for rec^nition for sucGessively longer periods. 
But determining where those dividing lines should be would require 
as nice an exercise of informed but subjectiv.fl Judgment as the 
.3,curren|! decision process without formal weights. No real reduction 
in the subjectivity of the decisions would be achieved* 

Adding the weights of each criterion an agency meets, to 
prdduce a .numerical scort that would determine the agency's recog- 
nition status, specifies a type of decision that may not be 
appropriate. ' It assumes that failure to satisfy any particular 
criterion can be compensated for by satisfying any other. Failure 
to require that the visiting team meet with all appropriate groups, 
for example, could be balanced by the existence of appeal pro- 
cedures, even though those two criteria have nb obvious relation- 
ship. Some other character istic ji however I such as requiring a 
detailed and probing self-study, might realistically compensate for 
the failure to have the visiting teams meet ^ith representatives 
of all interested g-roups • ' But compensating for one failure by 
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satisfying, a totally differeat kind of requirement is difficult 
to justify. The moit' important criterii might be considertd 
inviolate, with failure to meat any of them fatsl to the agency's 
recognition. 

The inappropriateness of letting satisfaction of any fcriterion 
compensate for failure' in any other can be partly avoided through a 
dual approach. For "exaroplBj 'recognition might be based on satis- 
raction of every criterion having a simplified weight of 3 in 
Table 25, plus jnough crlterik weighted 2 and 1 to reach a-speci- 
fied score. With weights based on importance alone , and excluding 
criteria weighted 3, 42 ^poirt&s could be reached by satisfying every 
criterion weighted 2^ or 1* A score somewhere above 30 for the 
lower-weighted criteria might be considered acceptable if all 18 of 
the criteria weighted 3 were satisfied* But this kind of process 
again involves subjective Judgments nd less difficult than those 
involvad in reaching decisions without the use of weights, an,d it 
ignores the interactions among qualities* The seriousness of the 
failure to satisfy any given criterion depends on an agency's othtr 
characteristics and its ability, in -vijiw of its strengths, to 
compensate for particular, deficiencies* But whether its strengths 
compensate appropriately for itm partlcuiar weaknesses is the kind 
of decision the Advisory Committee now makes, and replacing their 
j.udgment with a mechanically applied formula is difficult to 
Justify* ^ _ ' 
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CQmbining observabil ity with importance in calculating weights 
would reduce the effect on decisions of important attributes that 
are difficult to observe or evaluate. These include' the criteria 
concerned with the fairness with which standards are applied (bti), 
the independence of the agency's judgments (d1)j and the adequacy 
of. the agency's staff and procedures (g2i) and financial resources- 
(a2ii). Such a weighting scheme would separate the c^riteria 
concerned with procedurar requirements, which are readily observed, 
from those related to the effectiveness wiEh which those procedures 
are applied, which require the ^ judgments' o f highly- perceptive 
observers. Every criterion with a weight of 3 derived from both 
importance and observabrLlity was procedural. None of the criteria 
concerned with^ ic.tual effectiveness had i weight greater than 2. 

SUMM ..'Y 

Three sets of weights were calculated for 40 of the 44 
criteria^ excluding four that had not been judged for either 
importance or ; observ ab i 1 i ty\ One set was based on * the j udged 
importance of statemente that were either direct restatements 
of a particulirv criterion or direct etatefnents of a component 
of a criterion. When more than one statement veferred directly to 
% given criterion. Judgments of their importance were averaged to 
give a weight to that criterion. A^second set of weights was based 
on judgments of the degree to which compliance with each criterion 
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;can realistically be observed. The underlying assumption for 
thege wtights was that important conslderatiohs in the decision 
process should not carry a high weight if they cannot be accurately^ 
assessed. Finally,, a set of weights was provided that combined the 
first two. 

The composite weights were hiqh&st for the criteria that 
were bot,h important end readily observable. These were invariably 
criteria concerned with procedural ;Eequifements and organizational 
structure* Criteria judged important but difficult to observe were 
those that involved the effectiveness or quality rather than the 

form or structure of an agency * s activ it iea . These received 

-■ - - ^ • ^ 

lower composite^ weights. A few criteria, judged low with respect 

to .both standards, were glvfn the lowest composite weights. 

Deipite 'clear differences in judged importance among some 
of the criteria, the use of a formal system of weights is not 
desdrable for decisions like the recognition decisions. The 
r© letlonahips among the various criteria and their components, 
which/are likely to cause the desire j we ights to fluctuate with 
different ciTCumstapces , as well as' the absence of any explicit, 
single standard o f accrediting agency # f feet iveness , make a- 
decision proqess based on a fixed system of weights questionable. 
The eKliting procedure, in which knowledgeable, informed persona 
weigh each criterion in the context of an agency's particular 
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strengths and weaknesses , pre^^ferable. The criteria are impU- 
citly weighted in that process, but the weights are suited to the 
circumstances. Yet the problem presented by important qualities 
that are difficult to evaluate^is not av^oided by weighting them 
less heavily ttfan the important , but jeadiry, observable qualities, 
or by weightdng them equally with the observable qualities of 
comp^rat ively^ low importarice* The problem is serious, but it can 
be met real ist leal l^y .by focused efforts to define those important 
qualities in enough scope and detail to permit . effective evaluation- 
procedures to be devised. . ' . 



XI, IMPLICATIONS AND RECOMMENDATIONS 
The preaent recognition proceduree effectively discriminate 



among $ccraditing agencies with respect to a well-developed set of, 
organizational and procedural standards* Accrediting must follow 
prace^.lires that have been established by tn agency with enough 
clay 1 ind strength to protect its decisions from Ineptitude, 
carelasshesb » ' ifp4dequata bias, conflict of interest, and 

otheT threats tp sound and impartial evaluations. Most of these 
staridards siife strongly endorsed by virtually all. groups with. an 
iateres|j^^^W^ae^rediti (Chapter VI). They provide a widely 
accepted model for accrediting and state approval agencies that, by 
focusing on org inriat ional structure and evaluative procedures, 
oids prescribing educational content, Jstandardsj or procedures 
while encouraging sound" accrediting practice. 

'No evaluation of a process as complex as the , recognition of 
accr^^^nirig agencies can be adequately summarized, however, in a 
few simple Judgments. Some parts of the process will invariably be 
more fully developed than Qthers» and observations of particular 
rather thin global qualities will therefore be more informative. 
The i r.ipl Icat ions discussed below^ and the recommendations that 
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follow, poLnt out artas in which the pQBsiblUties are greatest for 
improvinij the reli^ility and validity of the recognition process, 
as well as the areas in which the procese. is particularly effec- 
tivt. ■ : ' 

The criteria in^ude in some form all the appropriate issues 



related to the svsluation of educational quality as that process 
was viewed by a wide range of experts* Despite wide variability 
in specificity and^clarity of definition of the criteria^ the 



recognition process was shown to discriminate consistently among 
agencies from year- to year, and the qualities on w^i^h thase 
-^-lic r imdnat ions rest were judged pertinent to the evaluation 
a f educational quality* The process is particularly strong in 

promoting sound administrative and evaluative procedures among 

^j^i^ ■ . , / ' ■ i ■ ■ , / 

"recognized agencies* ^ . \ ' a 

■ ■•' ,■ ■ ■ ■■■ ' -4 

~ - - . » . .. ; * 

EVALUATING THE EFFECTIVENESS OF PROCEDURES. ■ 

Several recognition criteria move beyond procedurar issues to 
imply standards of effectiveness that are not definTid, Accrediting 
procedures must effectively reject unqualified programs or . stitu-. 
tlons* Standards for accreditation must be applied with fairness 
and consistency* An agency's staff, visiting team members, and 
governing Xoard members must be competent, and its financial 
resources mu^K be sufficient for its purposes. All these, are 
important, accrediting agency qualities that are difficult to 
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evaluate. That such qualitiea msy require tlje subjective judgments 
of ■astute observers I however , does not reduce the obligation to 
evaluate them as accurately and completely as possible- The 
reliability of the recognition process^could be inereised by 
avoiding these difficult evaluative taeksi but its validity ^ould 
be correspondingly reduced; 

Qualities involving effectiveness or merit 'rather than 
the presence or absence of an observable procedure can be evaluated 
with acceptable levels of reliability and validity without avoiding 
the subjectivity that necessarily underlies their assessment, ^ The 
necessary requirements are clarity, scope, and detail, but mt 
rigidity, in the definitions of those qualities, and a variety of 
devices or procedures to indicate various forms of effectiveness," 
Meeting these .requirements is aot necessarily either complicated or 
costly, yet its accomplishment would add substantially to the 
' val idity o f the recpgnition process by increasing the empirical 
base for inferences about the effectiveness of. accrediting agen- 
cies, A major part of the recommendations that follow involves 
procedures for defining and evaluating qualities of effectiveness-- 
for accreditation, as .well as for recognition, where the need, may 
be as^ great , . 

THE STRUCTURE OF THE CRITERIA 

|bna form of 'validity requires evidence that the entire 
.range of agency characteristica. related to their evaluation of^ 



educational quality be assessed in the recognition process, 'The 
assessment mUst be complete. While the present study indicates 
that that is the case, the suggeated restructuring of the criteria 
CChipter IV) can help in periodically reexanining that issue. All 
the present criteria Tor recognition of an agency are encompassed 
in three broad areas--scope and.purposei organizational struc- 
ture, the quality of the personnel and financial' strength; 
accred it ing ^'and evaluative procedureo, including the formulation 
and application of standards* and responsiveness to the needs and 
interests of the'vardous const Ituenci^ , inc^'jding the general 
public. If issues relevant to the evaluation of educational 
'quality can be found that are not Included in these three general 
areas, the recognition criteria will have been demonstrated to be 
incomplete. No such issues were found, in the present study.. 

The components of each general area can also be examined to 
assure that they fully cover the area. ^Issues aboMt which there -is 
som& dispute can be evaluated to determine whether a proposed 
iadditlonal criterion, such as one r&quiring more specific attention 
to consumar protection, fills a gap in the existing structure, is 
redundant, suggests a.needed modification of the structure, or li 
not relevant to thfr evaluation of educational quality* Another 
kind of dispute/concerning existing crittiria to v^tilch objectioni 
have been raised,, can bs addressed by examining how comfortably 
the; questioned criteria fit the t'^ot of tna structure. In the 
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sugqested structure of Chapter IV^ for examplei including ethical 
practices and the encourigement of experimentation and. innovation 
under responsiveness seems forced. They were not clearly located 
in the ractor anaiyses and, at the least, require clirification 
and elaboration* 



OBSERVATION IN THE RECOGNITIGN PROCESS 

The nature of most of the criteria, with the exceptions 
noted above for those concerned with effectiveness, permits them 
to be readily observed and evaluated through documentary fvidance 
that describes the agency's structure and procedures * Observations' 
of visiting teams and commission meetings provide some evidence of 
adherence to the presbribed procedures* But substantial room for 
er.ror exists. An agency can be actuaily observed putting its 
procedures into practice on only limited occasionSj and the ex is- 
tencF'^^fis^approprT&te procedures doer not ensure their effective 
implementation. Expansion of the scope and variety of DEAE*s 
procedures for observing the per formance of accrediting and state 
approval agencies , in addition to their structure and established 
procedures, is likely to produce an appreciable improvement in the 
validity of the recognition process, although possibly at some 
C0Qt'4n the form of reduced relisbility, . ■ 

The most direct evidence of the effectiveness of an accred- 
-it ing. ag incy. is the .quality of its products or ^the consequences of 
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" ictions. Evaluative observations are beat diricfcedl to wherever 
•^j^^e products or consequencte can" bast be obSBrved^ . The^.self- 
jdies, the site visits, and the commissian meetings to eyiliiate 
f results of the site visits are probably where the conee- 



cfs of -n agency's ictivit its can best be 



obaervedi At pre- 



vTit, hey are sources of information for recognition j . but 'the^ . 
luld be exploited more tfully. Selected aspects of those three 
^ I itLss cculd be singled out for observatiorj and information 
ir-er^d from di f ferent participints . A moderately detiiled 
^dul5 of ..'valuative information to be collected from samples of 
ir^Lcipants in the accrediting fsrocess — ^agency ^staff, cofnmissiqn 
^ ^^r^, raculty and administrators of programs or institutions 
iluatad. ^tudsntSi members . of professional associations and 

T' 

i ^^'imq boards-^CDuld be followed without imposing a heavy 
.r Jen an particular individuals or institutions */ If collection of 
u evaluativft information were l6ft to the agencleS|\ its threat 
^ .Id be reduced and it would serve a function similar toothat of 

- if- studies of institutions or programs. .DEAE*'s function 
.\ i t-^n be that, of an auditor of the agency's stflf-evalu^tive 
^ir-^ation* Thiy possib il ity ' and the other issues described above' 

- irMres^ed more fully in the recommendations. . \ ^ 

^ "MFSDAFIONS FOR DEAE ..Ij. . ' - ^ 

An iTiportant purpose of the study waS' to develop, recommenda- 
for procedures that the Division^ of Eligibility and Agency , 
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Evaluatian might ^^low in a continuing prog^ran of sel f-eviluation. 
Its focus would be the iuitibility of the recognition criteria and 
thai effectiveness of DEAE's evaluative procedures. Since the 
crittrii require that accrediting agencies evaluate their etandtrds 
and t^e effectiveness with which they are applied, DEAE might 
reason ably 'be expected to demonstrate that process in its own 
activities * While the present itudy produced evidence of relia- 
bility and validity, the recognition process is not a\d should not 
be static. Continuing or periodic reeyaluation is impoptint* 

A project of the magnitude of the present study would not be 
feasible for most accrediting 'i^genciea and coul^d noi be carried 

out at frequent intervals by DEAE. In that sense it /cannot serv-e 

t ■ . j 

directly Bm a model for DEAE or for accrediting agencies in evalu- 

' • ■ ■ ^ ■ : ' 7 . ' ■ " ' 

itlng their own standards and procedures # Yet problem areas, have 
'been identified, evllgative conlcepts and princl/ples have been 
.demonstrated, and selected aspects of the procedures of the present 
study can be applied both by DEAE and accrediting agencies* The 
follpwing^ recommendations were formulated w/th. av^reness of the 
limitations necessarily imposed by cost considerations. 

Re liability • . ■ ^ 

, ' * . . ^ ■ " _ • \ . " ' ' \ 

As was. pointed out in^^Chapter JI, reliability is the absence,/ 

of tan^dom or -unsystematic ferror in a proceaSi It is ^indicated by 

-the conir. latency of ^he rraults which can be evaluated at the end 



of B procesi i§- well as at a number of Intermtdiate points. The 
degree of consistency in the end results of a procest indicatea iti 
overall reliability. But consiitency in the final product may be \ 
more difficult to observe than the consistency of several inter- 
mediite or contributing processes, and the reasons for the error 
or inconsistency may n&t be evident in the . final product. An 
effective program to evaluate and. maintain the reliability of a 
process must therefore be directed to each^step in the process,,, 
where error might enter > 

^Wrth respect td the recognition process, error can be intro- 
duced at the point where observations are made, at ithe various 
points^ where the information is transmitted or transformed (as from 
,the obeervat ion .itself to the teport of the obsarvation, or from 
ths! Veport to its consideratiort by the Advisory Committee), and in. 
the final del iberat ions lead ing to a decision. The presence and 
extent of the error can be estimated through observationa ,of the 
consistency of the information. ^ ' 

' Consistency in the initial gbserv at ions can be assessed by 
using more than one observer and cbmparlng their observations* 
T^his does not necessarily require independent visits to an agency, 
by two or morr observers and independent reports. Such a procedure 
^wduld be costly and not the most effective way of identifying 
d If feirences in tht observations*^ ' A more produGtive process would 
have several observers make a moderate number .of observations about 




several Lespiicit aspeets of an agency. For example, several 
critifia require agencies to publisH information about^ themselves-- 
a" clear ststiment of scope and purposes, explicit definitions. oZ^^^^ 
each accreditition status, current standards and decision pro- ' 
cedu^ei, effective procedures for review" of complsints, and othiriV 
Each of these criteria can be evaluated through an examinition^ of 
agency documents . The rel iabH ity of those observations could be 
assessed by gathering^ a sample of documents related to fich of 
severar criteria and'' asking DEAE staff personnel and^ consultants— 
those who^ regularl y make^ the observ atlons"to evaluate, the sets 
of documents. Others, such as faculty members, adm.4nistrators , and. 
fgency directors might also profitably beasked to make thi^ 
judgments about the same documents. Consistency of observations or 
judgments among DEAE's staff members and consultants regarding 
several kinds of information about several different criteria could 
then be assessed, , . 



^ Aoreement or consistency'' might be high with respect to clarity 
of scppe but less impressive with respect to the adequacy of 
the published decision procedures. If agreement was low for -a 
particular criteribn, steps could then be taken to identify the 
source of th^ disagreement, which may lie with the observirs, .with 
latk of clarity In thei criterion, or with^the documentp* Whether, 
agreement was high or low, an index of interjudge agreemrfrit, such 
as the kappa index used in the present study, could be calculated 
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to dBScribe the rtliability of the collective observations of all 
the criteria Judged,, as well as the .reliability of the Judgments' 
of eaph separate criterion. , , . u 

Two points in the preceding partgriphs deserve particular 

emphisls. 'First, thi intent of the process .described waslo focys 

on a limited group of relat'ed qrlteria rather than on one criterion 

or-many. Limiting the effort to s-tudying a few criteria at a time 

keeps the cost flianageable. Over a period of sey,eral years, all the 

criteria can be eyaluated. 

. . -. ... ^ . , ' : ■ ' 

the 'evalu'at'idn of' several kinds of observations or several 

criteria, rather than one at a time, is desirable , not only in the- 

Inte'rest- of economy, but also, because the degree of consistency or 

reliability is likely to vary.. Assessing the reliability of 

several k inds o f obsefvit ions , provides a context in which the 

rel lab il.tty associated with a pingle kind of observation can be 

evaluated, further, if several criteria are reliably observed 

while one or , two are not,' the inconsistency may suggest the naturr 

of the difficulty with the criteria that were not reliably db- 

served. In the present study, for ex ample ,: the differences in 

observability b|tween qualit.ies reflected simply in the existence" 

of appropriate documents or procedures, 'and those related to the 

effectiveness of an^, agency's actiyit'iea, pointed out a-major 

distinction between crlterii. ' The first kind'are inherently 
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observablej the second are not. ■ ,, . , 

■' .'. " ' ' . . ^^ ■ / ■ ■ '-i .■ 

The aeOOfid point to be atresstd is that the procedure de- 

scribed aiaesses the reliataility of . only one important aspect 

of the recognition proce.sB--the obaervationa thefnielvei. Jheir 

meaning may latigr be diatnrt^d i or ti^ror introduced, when aumma 

rized in a written report , when discuasid in an Advisory Committee 

heating, or when combined with other information in reaching a 

decliion about an agency* ' . . 

The reports are usually the result of the collabopitive^ 
efforts of Be.v.eral DEAt staff members , and are written after 
discussion of the observations with the person who has direct 
responsibll ity. for the report. Variations in perbeptioni or 
emphases are thej-^fDro raducrd in the process of waiting the 
report. Systematic? ^ijnrs or distortions of the "true"' state 
of affairs may -t^ may not be affected by collaborative report 
writing., but tKos^ arrara will^ affect the validity rathtfr than 
the teliability ef thi-^ proceaSi If the consistency of thp process 
is incraaaed, LI teliability will he increased whether or not the 
consistent information is gorrect. . 

Errors in consistency in th§ ronD.^t-writinfg pr'^ctas could be 

^' ' ' ■ , ■ 

both a^iseased and reduced if two groups, of DEAE staff members were 
to revitw the same observational dati and produce two independent 
reports. In the IMerests. of Ppecificity, time, and cost, hqwevtr, 
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- only lifflitod fio^ctlons of repbrts concirned' with a-'smill gr^up 
bf'crltfrlt might be treated Chli way.. The dtfffettncei in the 

■ * apaeific stattmtnti composing the. two reports wouidVindicate the 
digrge-cF ^^jreenitnt and the^nsture of the disigreementi* One luch 
comparlatn each fnonth ov^r the course of a^ ytar would Increasa 
the ov&raJl relUbil'ity of ths reposta, as a v^ole while providing 

; . aii estL-naU of its Itvel, ■ ' - ' 

■■■ ^ ■ ■ ■ ■ ' ' ■ 

^ tne filisbility of the dtcision proceei could be ^similarly 

asaejtt^J* The Adviaory .Committee typically conaidere petitiGni. 

^'S;' for .re^-ognltion in. two iubcommittees meeting simultaneo'us>y', each - 

\ hearing arai prsaentationa by ripreaentatives of 'the ageno>r under 

:'- . ' , , - ■ ■ . / ' ■ . ■ 

' consildaration, and by other interested perioris who may offex 

. ^ ^ r ^ . : : / /V r ^ / ■ ■ y ■ 

infor'^'it f on. The two subcommitteea consider separate gTlroyps of 

/petltior^s and recommend decisions to the full Committee',: which then 

; vute0 un a decision tHat.ia redbmmendefl' to the''Commislioner * Whlla 
^ ^ ^ . 'I ' - ' / ... - ■ . . •- 

/\ th^ creatfbh of the new Department of Educ^ hat aboliahed the 

office of CommisaionerV the furtction of the Ad viaory Commit tie ; 

'= in recommending a decision has 'not changed. The reliibility of the 
Advisory Committee's actibne could be evaluated fay havlng^two or 
three .petitiona-conside'red by'both.subcommitttes at each of their 

. ' .qur^-terly meet iriga , / Aa with the written reporta, letrning the 
— -^reasah for any discrepancies .'would lead to an improvenient in 
reliability, and the eight or twelve decitlons replicated during 
the course of a year would permit an estima'tion of the overall 
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.level ofreiiibility of the Committee's recommenditfons, 

" ' ^ ' - ^ / ' . /• ' 

Alternatively, the subcommitties could be split for the dual 

cohsiderition pf two or three petitions at each quarterly meeting. 

\ ■ ^ .■ - : ,^ , ' ' - /' 

That arrangement would not add to the Committee's work load but 

would^ tempor'arrily leparate two three- or fqur-person groups within 

■a subcommittte Vhile they coriaiiereB" the same petitio Their 

independent decisions wbuld be Recorded tb assesi the reliibility 

of the process, and then th&lr Joint e^^amlnat ion of the two 

decisions woulA lead to the recommendition to the full committee. 

While the time required for consideration of a particular 



pet it iqn might be increased slightly by hav ing ■ two .sub-subcom-^ 

' . ' ' ' ■■ ' 

mittees discuss a petition before they recombined for the recon- 
ciliation or theLr differences, that might not be the case* 
Three- ^jr fou^-person groups might reach decisiohs more quickly 
than, i seven- Qr sight-person group, and if the decisions of the 
two: smaller groups agreed^ little ^f any time need be taken in 
further^dal ibaration. During the, course of a fouf-meeting year^ if 



two. petitions per meeting were considered separately by independent 
halves of the two subcommittees, a total of 16 recommendations 
would be avillabli for estima^tion of th6 Committee 's reliability, 
th re'm petit ions per ,mee|lHg considered by indepei.dent halves of 
the two subcommittees would ia a year produce 24 cases /from^T#%4eh ' 
to estimate reliability. - ^. ^ 




If ixpetltnce shQuld show that tht ^hrte- or rotir-person 

groups produced rtllabla decisions, thty might than be used 

reguiarly ^for the less camplex or the less controversial petitions 

the Ciommit tee must evsluste. With the Comm it tee' s work load 

continuing to increase , the / uae of fopr rathe^ thin two subeom- 

mittefes cbuld be an effective way to increase the Committee's 

efficiency and handle the increased load^ ^ : ^\ 

^' . . ■ ■■ ■■ '■ ' 

The reliibility with which information .is. taken frorn, agency 

documents can be assessed by comparing two or more seta of inde-. 

pendent obBervattons oP tha same documents. Although! KeroK copips 

o f ^on-site . V isits or eommialion meetings are not yet availabres 

the replication of obaervations in the fifeld can be carried out* 

even if at somewhat greater coat than the duplicating of documants* 

Observations of two_ v Isiting teams or two commission meetings or 

two other agency settings could be^ mada by different DEAE staff 

members at diffarant tlmes^ and/thair results com^paped. Again, to 

keep the cost and the. burden on the accrediting agency reasonable, 

the r,epeatad observations should be 1 ynited to particular aspects 

of the criteria, such as evaluation procedures or fairpess. 

The twb sets of ob'servatlone should oocur within three or four 

weaks;.of each other to avoid^ ttie; possibility of changes in agency 

procedures from/ one ^set of observaUo'ns to the other. If the board 

or commission/only meatr at six-month intervalsi the two obsarvers 

might attend the same neetir^ on different dayi^.^ 



; Dull observitions- of tHt « limited set of criterii at eight 
or ten different agpncies during the course of a yeir would prdvlde 
a rtalistic- set of dita from whichlto tstimita thp reliability of 

^,t!he more difficult ob^rvatipns.. Differences in the content of .the 
observers' ^ reports about; the isme agencies could, result either from 
dif ferences in the observers' perceptions or frbm ^differences in 
the particular activities observed, Cither sourc,e of inconsistency 
would reduce the reliability with which that particular criterion 

^was. being evaluated. Regardless of the sourbe of the errors if .the 
reliability was too low for much confidence to' be placed in the 
accuracy of the evaluatiQn--that is^ if the inconsistencies^ were, so^ 

qreat that the qual ity o r aiT^ihcy:^wl:th^espect^to-tha 

.... ^ - ■ ■ - ■ " . ■ . i , s 

istic could not. be stated with confidence— some procedural change^^ 
wpuldi be requirjed. The most llkely steps would be to clarify the 
erratically observfed criterion and expand the observations related : 

Whatever the sdurces'of unreliability in the recqgnition 
pro/cess-.-whe ther the observations themselves are erratic or 
the in format ion >they prpvide shifts as it is, moved through tlie 
prpces's--efrQr can be reduced through the jjse of greater nuinbefs 
of observationa • Fai r nes*"^ i f o r ex ampl e , m ig ht b e observed .in 



le assignment of visiting team members! in the initensity, st pe, 

iv .thoroughness of the visits in tl^e apppointment of governing 

^ . . . ... - 1 

oard me^bersi or in the implementation of a reguirement for the 
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evaluation of student learning. Observationa that raised questions 
about the fsl^nesa or\integf ity of in; agency in sev:irai of thsfB 
lir-ias ^would be fap. more serious than an observation of an isolitid 
questionable practice.' Each of the exOTfjles/ibove is aasociaked 
;ih part with a different criterion, but feach can be applied as 
well to Criterion b8, v^lch is concerned explicitiy with fairnese 
aKd Impartialip in most appeDti of an agency's activiti^Sy its 
"stand.ardi, policS^i^ procedyres, evaluations, arid deciiions. The 

qualitiii most difficult to pbserve--fairnes3y e f feet iveneis , 

^ - ' ^ - . ' ^ " _• 

_and conipetence,; for e5<imple--ari those that should be the focus of 

; ' ; - ' \ ^ ^ *\ ^ . .... . ,^ . ^ . . 

^ a wide range of specific observationa , The present study indicated 
^^TT^Tm eTio n ^b^^ la 2 i and^ a 21 i on 

the adequacy of staff and financial reso^ces were" among the most 
'icnpbrtant critefla and among the most difficult to observe. They 
reguire a wider range^of related observations if the reliability 
of th"e^^^^evJl^ is to be brought closer to that of the more 

observable criteria'; \ .\ ^ . . ^ 

Racofnmendat ions for Evaluating and Improving Reliability 

. ^ The preceding discussion leads' to the follDwing recommended 
procedures for DEAEi 

1i Establish review committees tontposed of OEAE staff, 

f 

accrediting agency pirsopnel, and others to review the 
criteria most d if ficult to evaluate (starting with those 
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liit&d iimediitBly below) md recommind idditlonai observ-, 
^blB _ indicators of ageney per formance pert intnt to thosB 




b8. Fairness, impart raTity'--___ 

. a2i. Adequacy of staff and^ procedures 

d1 . Indefiendence of Judgment- 

aZii. Financlad strength and independence 

Separate the criteria into sets that are simiraT in^ 
content and.flegree o f observabilityi establish a regular, 
system through which the reliabilities and validities of 
all the criteria are evaluated over a four- or fiv^-year 
period^ one set at a time* 

Establish a regular process of d^al observations ff 

documents that bear on selected sits of criteria*^ 

■ ^ ^ " ' ' 

Esj^'abllsh a regular process through which dual obser- 
vations in the field are made of accreditihg agency 
practices related to sfelected seta of Mlteria. 

for those criter/a thatjShow low consistency of obser- 



vation, establish review committees of DEAE staff membersV 
other interested perHonst or both to study ths ngjtura of 
the disagreements and suggest clarifica^innj elaboration, 
or reviilon of the crit./^riai improved trajnlng of the 
observers; clarification of guidelines to the ^enciesi or 
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additiqnal sources of information pertinent to the erratic 1 
observations* / \ ' " ' I 

. ■ ' / " ' / - ^ • ^ ' ■ • -1 

6. Establish a regular pattern of dual report preparaticri 

' : : ^ ^ ^ ■ ^ ■ ^ ^ ■ - ^'^ \ 

fpr selected sets of criteria to be follQwed by study and \ 

clarification of the areas for which consistency was \ 

■. ■ - ■ . - ,^ - - ^ . . • . ^ 

^ low. 

7. Assess the consistency of Adv isory .Committee activities . 
^ by having two or three petitions per meeting evaluated 

independently by both subcommittees ur by independent 
halves of the subcQmmittees ^ 

yalidiry^ 

While reliability involves e search for consistency, the 

"■ ■ . ^ ■ ^ i / " * ' ■ ■ . . J ^ - x 

■assessment and improvement of validity require the development 
of empirical support for the desired- in ferences. For example, 
satisfaction of Criterion bS implies that an^ency carries aut its 
actions fairly and impartially , Evidence of impattiality that is 
Independent of the information that determined the satisfaction 
of Criterion bS would support that" impl ication # One way^to demon- 
strate valiclityj therefore, is to show agreement in th^ results of 
several/independent sourcea of evidence for each critBrion. 

A second asptch of validity is the completeness of the sup- 
^portable inferences. If fairness" is td be inferred, all of its 
componeriti'shouid appeiE in the _pertin^nt evidence. Observatiuns 
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related to, one component in a single content cannpt adequately 

_ -/ • ' ; ■ . ^. , i 

suppdrt inferences about fiirnass in general. The validation 
of a process therefore requires s-ome attention to the^ scope and 
r^ge of appl icat ion of the pertinent evidence. In the present 
st-ydyf almost all. the criteria were reflected in the recognition 
decisions, ^ut the cc^ltteness of the evaluation of each criterign 
Wis not .clear , _ Fui ther^^elaborat of -some of the criteria, 
' particularly; tHose difficult to observe, may indicat^certain of 
th^ir/aspecbs that have not been regularly ivaluated. 




/ -7^ third, way to support the ihfecences/'drrwrT from any p 
to compare the results of the process with th»ose Q^£-4Tidependent ^ 
activities that had a similar purpose. Attention" would be given^ to 
|.he,prqcess as a whole, rather than to ifrr^omponehts,v? That UindxoT 
validation^wa's illustrated in the present-study by'^the gro^ of 
judges who" prov ided their own recognition decisions .for compafison 
with- the actual decisions. ' 

• / ^ :^ . - ^ - 

■ DEAE ^might carry G.ut a similar process ever>^'foqr or \five 
yeara, checking a sample of. actual decisions ffith those made;by. 
a selected group of persons whose knowledge of accrediting' is 
respected/ The nature of the activities of the validating group of , 
experts .would determine whether the entire ^roce^ or only ^ part ^ 
\of it were- to be' v al idateds 1 f the experts were provided just. 

the informatron availabie to the Advisory Committee— the agency's 

- ■ ■= ' ^ 

petition, the DEAE staff report, and a transcript of the hiyring 
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before the Commi ttee'-- the validation would refer only to the 
Adv iflnryvCDmniittee ' s actions^, evidence would not have been pro- 
vided as to the accuracy and completeness of the information frofn 
which the^ Committee worked. Tor the entire recognition process to 
be validated t a cnmplete evaluation, independent of che DEAE 
evaluation, would be :eeded for a ^ample of agencies. Such a 
procedure would be difficult, expensive , and probably unjusti- 

fiuble. / I 

I ' ■ ■ 

A stTipl«r ^ay to validate the avirall results of the recog- 

• '^i ' . ■ 

nition proctss would be to compere recent recogni't-on decisions 

with olher indicators of overall agency quality. A variety of, 

limited indicators of quality or effectiveness could, in the 

aqqregate, be expected to reflect the general quality indicated by 

recognition. One such indicator might be the proportion oT tfte 

toAar population of eligible institutions or programs that seek 

accreditation from an agency. Another might be the proportion of^ 

unfavorable dec isions .that are appealed * A ^ hira might consist 

of coffiparisonci of programs or institutions accredited and denied 

accreditation with respect to some pertinent index, of quality. 



Even these limited examiriations o f qualities, associated with 
.different recognition decisiona are. likely to be more elaborate 

and a^c^pcnsive than their usefurnsss may Justify, Accrediting or 
, state appMval igericies and tha issues with which recognition, is 

conGerned are so complex that gross c^pariaoris of ^encles recog- 
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riized and denied recognition are not likely to be very informative* 
A more productive and more efficient prodesa would consist of a 
series of intensive evaluations of limited aspects of the recog- 
nition process I applied to a small sample of agencies, and cumu- 
lated over a period .of four to six years* By the end of that 
period, changes in accrediting, in postsecondary education, and in 
the nature of the public concerns would probably make revalidation 
of some aspect of the process desirtble. f 

• / ■ ^ . 

Alternative Sources of Evidence 

' .Feasible sources of information^ or evidence for continuing 
validation of the recognition process include the petitions and 
other documents submitted by the ^encies, the direct observations 
made hy DEAE staff a-^^ consul tants , the attitudes and opinions 
of others as presented in correspondence or in direct testimony 
before the Advisory Committee, and attitudes and opinions solicited 
from persons who have had pfjr^ticular kinds of experiences with 
accreditinq* Each of these sources except the material from the 
Advisory Committee meetings w^as used in the ^present study, and each 
could be used in future assessments of validity. If two or. more 
of these or other sources of information were to lead \to similar 
inferences about a particular criterion, that ag reement between 
dissimilar kinds of information would pruvlde empirical support 
for tht inferences to which both kinds of information led. The 
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following discussion of the criteria related to fairness illus- 
trates the point. 

FaiTness, in its several contexts, refers to the application 
of an agency* s established^ standar ds and procedures , /the imparti- 
ality of its decision procesSj snd the absence from its accrediting 
activities of conflicting functions or competing/ihterests. The 
:nost common sources of information about fairpass are (1) documents 
provided by the agency that describe functions and proredures 
designed to reduce the possibility of bias, and (2) direct obser- 
vations of the agencies* act i..\rlt ies by DEAE staff members or 
consultants- For example ,^.^^rlt ten procedures that permit board or 
jommission membe-rs t^o^participate in or influence decisions about 
their own institutions constitute documentary evidence of the 
possibility or b i a se d dec is ionsx* Simiiarly, an observation 
in a oommission meeting that qualities not pa^t of the published 
standards had influenced an unfavorable accreditation decision 



would also constitiite eviderce of unfair application of standards, 
\eithe^/of these observ at ions , In the absence of further infbr- 
mat ion I would justify a conclusion that the ^ency had exhibited 
general lack of fairness. But the evidence from the documents 
□ lone would be more cr ivincing if the participat ion. of c^ommission 
mpmbers in decisions on Lheir own institutions were more frequently' 

observed in agencies without clear procedures to prevent it. And 

'"^ • X ' / ■ 

if observitions that decisions in board or cofrtrnlssiorr meetif^s were 
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being based on issues that did not ippear in published stindards 
were often accompanied by similar reports frofn constituent pro- 
grams or inatitutions, the utility'o^ such observat ions would be 
conrirmecl, 

A third source of information, in addition to the agency's 
documents and direct observ &t^ions by DEAE personnel, might be a 
^ody of obsgrvat^ons collected from programs or : nst i tut ions 
evaluated by the agency in recent yearst They could take the 

\ 

forni of a short checklist of possible instances of ^ency behavior 
that eitner . demonstrated fairneBS or "raised questions aj?^6ut it| : 
foirowed by phone calls from DEAE for amplification. Alterha- ' 
tiyeiy, a letter to from 20 to 3D"recentliy evaluated programs 
outlining tf^e nature of the concerns for fairness, followed by a 
celephone interviewp would prov ide similar information. However 
collected, the information should be limited to ^the issues of 
'immediate concern and to a small number of aqencies'. 

^ ' i 

Comoleteness of the Evidence / ' - , j 



.i 7 Inferences about quality^ of performance or effectiveness are 
of^en challenged on th^ ground that' they are based on incomplete 
evidenbe, that the observations are peftinent only to small parts 
of the isssue of concern. A high degree of effectiveness in' 
limited aspects of a complex process, such 'as accred itation , does 
not necessa^fily imply a high level of perfdrmnnce in general* 
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Expert judgment is the usual basis for determining whether an 
evalyative process is sufficiently comprehensive. The recourse to 
expert judgmF''*'. ^ however, should be systematic. 

\ _ ■ 

EvaluatinL; he completeness of the content or scope of a 
process can takt- complementary forms* In one, the evaluator 

examines analyticali ■ cdverage within defined boundaries, 

searching for gapi. i 5 other form, knowledgeable persons 
3 'vvlne the predefin :-" - ^^1.. les to determine whether they ntm 
ovBT-^ .dluslve or not ^^'Ci J^i e annughp Any redafinition of the 
bour^'*sv ^ a'round the pertinent issues is,^ then' followed by a new 
analywp^ of the rederined areas* Roth' these processea require "the 
judf^me^tc qf experts, but hQwever campstent the eKperts may be, the 
quality of their judgments will be improved through prior inalysis 
and care fub presentation of the issues to be judged. 

A third k?.nd of expert judgment may usefully follow the first 
two. T>^:e relevance of the observations made,' or of the infcrmation 
collecLed, requires demonstratif^'n. For examplei a -mplete catalog 
of ail the issues related tn the evaluatioh of educatinnal ^fluality 
and nom inal . attention to every item in the catalog do not or*' them- 
selves assure adequate cdveraqsi the particular observations of 
each item must be pe"tinent. ihe determination of pertinence may 
Test partly on expert JudgmOTt, buv »,he convergence of two or more 
independent sources of ^evidence leading to a comhiun inference is a 
stronger base for Iha determination of validity. 
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In the evsluaUon of crUeria related to fairness, about '50 
somewhat distrhct dr ^criptive statements were used by DEAE staff 
member-i during tht 10-;^ear period from which reports were analyzed. 
They ranged widely in specificity from "No more than one member of 
the policy- or decision-making body may be affiliated with the 
same institution" to "fhe agency's published procedures are fairly 
applied." A small but varied group of knowledgeable persons 
revifewinq those 50 statement^ could Judge the adequacy with which 
they covered all aspects of tr^e four crib'ria related to fairness. 
They might also suggest additlQnal ways in which the quality of 
fairnesD in an f,^gency'& operations shauld appear, Evesn-handed 
tifeatment o f^ objections raised ty institutions to particular 
visit ing -team procrdur^B^ or vari:itions in the rigor with which 
particular standards were applied^ ars e'xamples of agency per- 
formance ^ -th re^oect to fairn^fss that tended not to appear in the 
DEAE sta/f reports ^ ,.A variety of cthrr ^j^ampleSj with suggested 
pfdcedures for observinwD them, could be produced by a small group 
or people who gave their attention exclusively to manifestations of 
fairness ahd unfairness. 

Recommendations for Ev al uallnq and Improving Validity 

The moat effective opproich to /maintain/; j the validity of 
the recognition process is likely to be a combination of the pro- 
cedures described above--brlngihg indepen-ient sources of evidence 
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to bear on a particular quality^ and evaluating the cQmpleteneii 
wi'th which various issues are assessed. The first evaluates the 
fidelity with which each important attribute of an agency is 
described. The second provides evidence that the prQcess is 
directed Co the proper issues in the evaluation of educational- 
quality and is sufficiently comprehensive* The following activi- 
ties are therefora. recommended , in addition to RecOTmendation 1 
and 2, listed earlier under Reliability^ v^ttich also apply to the 
evaluation of validity, ■ 

X 1. Using a sample of 10 to 15 ^encies that in prior evalu- 
ations varied in their effect iveness with respect to a 

, limited group of criteria , intensi fy the observations 

/' ■ ■ 

pertaining to those crllnria thraugh one or more of the 

following procedures , ^ . 

a. Gather amplifying documents related to the criteri-' 
being evaluated, yn addition to the petition and the 
supporting docMfnents that accompanied it. These might 
include, from the agendy itself, lists of visiting 
. team- memberships for the preceding two to five year.s, 

guides for progtama pr institutions , visiting team 
reports, and^ecoKda of actioni^ taken during the 
preceding^'^^ew years* Programs nr institutions might 
be asked for copies. of self-studies, reports of agency 
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decisions, or responses to the agfncy' a reports or 
dec-isionr*. Some documents may be confidential, hut If. 
the requests for documents are limited and specific, 
a/, they should be, problems of confidentiality and 
accessibility may be ivoided* Specified sections 
of a report I for example, with institutional identi- 
fication deleted, mny be provided when the full report 

would not be, 

i " . . 

Provide ah opportunity for DEAE personnel or cnn- 
suit ants to observe the circumscribed groups of 
criteria and agencies^ more intensively, either by 
exjtepding the time the DEAE obperver spenda on 
those qual'ities or by having a second observer 
concentrate on them* 

Solicit reports from samples of programs or institu- 
tions recently evaluated by the ^encies under obser- 
#^ 

vation regarding Instances of agency performance 

..^ " 

relevant to the criteria of interest. These should 
focurf explicitly on the stated cr iter ia'' and should 
ask onjy .for the experiences of that program or 
instituticn, A second stage might follow. in which all 
the instances describerf^fay the apmpled programs or 
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institutions would be sent^ in the forni of a che'ck'- 
list, to a larqep sample of agencies to determine the 
frequency ^f each type-of agency performance* 

d. Collect in forrration" on potential correlates of agency 
effectiveness that may be provided by sources other 
than the agency or its asaociatL . . rograms or institu- 
tions* These other sources might include, but are not 
ILmited toj licensing boards, other state agencies, 
^d other postsecondary educational associations, such 
as the Council on Postsecondary Accreditation or the 
American Council on Education* 

Evaluate, through one or more of the following proced^ures, 

the scope and meaning of the indicators of performance 

associated with the groups of criteria referred to in 

Recommend^st ^ons 1 and 2 under Reliability, 

/ ■ ^ 1 . 

a. Ask a s e 1 ed t e d/ g r 0 up of 50 to 100 knowledgeable 

S/ persona to indicate, on a short questionnaire, the 

Importance of &^ch indicator of agency effectivenesa 

to one or more of the general asaues the criteria 

' r ■ . _ 

address. The list should consist of 20 to 40 observ- 
able agency characteristics that are, in effect, to be 
sorted into categoriea according to^their d^rse of 
importance to each of the criteria of interest* One 
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procedure for accompl iihing this, though not the 
only one, would be to list each characteristic on; a 
cardp provide an envelope labeled with each of the 
four or five criteria pf interest, and ask the Judges 
to pltfbe each card in the most appropriate envelope* 
The judges shSuld have the optio,n of not puttino a 
card intp^ any of the ehvelopes if none seemed appro- 
priate. The judges could then assess the importance 
of each characteristic .to the criterion with which it 
wa? associated. Finally, the judges could be provided 
blank cards on which to ad^d characteristics pertinenb 
to the several criteria. The result would be judg- 
ments of the adequacy with which eKistlng evaluation / 
procedures cpver the criteria. 

Select 1,0 or 12 agenciBS that differ in the degree to 
which they satisfy qne or more of a, small group of 
criteria. Send each agency the statements related 
to those criteria with which it was described in the 
most recent DEAE staff report, with the observations 
on which the statements were based. Ask the agency 
director to indicate the accuracy of each statement, 
its imp'ortint'e to the criterion with which it was \ 
associated, and other observations that would have 
improved the evaluation of that agency .with respect to 



that writer ioa# The additional observations suggested 
should then be evaluatsd through some procedure 
similar to that described above in Paragraph ^. 

c. Rspaat, the procedurt described above Par^raph b^ 
sending the sets of statements ■ to the accrediting 
1 i a i s on 0 f r i c e r or other per'son. concerned with 
accrediting at a small sample of programs or institu- 
tions recently evaluated by each of the agencies 
identified in Paragraph b*. 



The recQni^endat ions made for both reliability and validity 
focus the effort on limited aspects o f the reGpgnition process 
while spreading it. ovar time. Although this keeps the. cost and 
t i^e requ iram.ents as light as possible^ it does reqtjlre the cooc-^ 
dlnation of the activities over a period of time for the results to 
c^umulate into a comprehensive descriptl^on* Reliibllity and valid- 
ity are both cc^stly to evaluate. ^y require intensive observa- 
tions. Yet without evidence to deffionstrat.e consistency and support 
the inferences drawn from the results. □ f an evaluative process^ the 
prf ^ess itself is hard to justify. The focused, cumulative 
approach recommended abdve minimizes costs while providing a sound 
foundation^ for 'defensible^ evaluative jadgments. 

, ■ . . .> - ' - ■ ■ 
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RECOMMENDATIONS FOR ACCREDITING AGENC4ES . 

The recognitiDn process 'is similar in many ways to accrediting 
and Btate appro.val. The, petition for recognition is similar to the 
seir-study of a pcogra-n or institution. The visit by DEAE staff is 
similar to a site visit. The Advisory Committe*«'a role in the 
recognition of agencies is similar to that of commissions in the 
accreditation of institutions and programs. 

The limitations of the two processed are also similar" though 
more severe for recoqnition than for accrveditatiqn. For example, 
the criteria for recoqnition may. not impoie educational or eurric- 
ular requirements dn agencies, and ^the educatlnnal aubstance of 
the progrpi or institutions accredited may not be circumscr ihed , 
The regional accrediting commiisions are similarly constrained 
in their attention tu tjie process, r athei than the . substance of ^ 
education, although a concerrt for standards cannot always be 
clearly .separated from a concern foi educatirnal substance. The 
'Specialized agencies', being more direjtly involved witH educational ' 
content, have more complex problems with reliability and validity 
tijian do the regional agencies. They must justify curricular' 
^quirements thst the regional agencies avoid. . ' — ^ 

! ^ The propedures illustrated, in the present studyj and^ the 
■recpmmendations made for evaluating the reliability and vaUdity 
of the recognition. procesi, can be uaed with some modification and 

2SG .... 



276 

i " ' , 

elaboration by accrediting agencies, and in fact some have.done 
^30. Each rtcoqnized accredit inq or ^.tate approval aqeney must have 
clearly defined standards* Establishing empirical support for the 
infererices to be drawn f^om' whether the standards ar^ satisfied, 
gnd for the consistency with which the standarda are applied, 
liay be accomplished in much the same way as for the recognition 
criteria and procedures. 

Doth reliability and validity are improved when standards 
.Tire clear In meaning and scope* Without c lar ity o f meaning y the 
observations from which performance can be inferred are di fficult 
tO' identify* Without well-defined boundarieSi the comprehensive- 
ness of an evaluative procedure cannot be established, A first 
step in an agency 's evaluation of its s tandarda may 'well be to 
i/ientlfy the nature of* the evidence on which compliance is' based* 
Review of the intent of a particular standard, and of the cistomary' 
procedures for Its ev aluat ion , by a small group of experts may 
then reveal important aspects of the standards that are not well 
□ Finessed* and may suggest additional kinds o f observations likely 
to be useful for one or more of the standards. 

Most accrediting ^encies carry out self-evaluative procedures 
tha t 'approx imate the process' describeri abovet Theii procedureii, 
however, tend to be less systematic than is necessary for confi- 
dence to be placed in the results* Moat groups of experienced 
faculty members and administrators co^uld suggest procedures, that 

■ s , ■ 
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would be acceptable indicators of educational effectiveness, and^ 

many agencies have implemented the recommendations of such groups. 

A group of experienced persons, for example, can undoubtedly 

specify a number of ways to evaluate whether the instructional 

* 

objectives of a program are being achieved, %e training and 
experience of the faculty can be reviewedj the formal "ofgani^ation 
of the instruction, which defines the nature and scope erf the . 
contact between faculty and students, can be exanined; the tasks 
required jf students^ and the procedures for evaluating student 
perfprmance, can be observed; and examples of student performance 

^ in, advanced levels of the program, and achievement in the activi- 
ties for which the program prepares students, can be evaluated. 

.■But until the^ consistency and implications of the specific obser- 
vations are established ^ judgments based on those observations 
remain speculative. 



Beyond the general^^meaning of observations, their variation 
with different circumstances must be understood. In some programs, 
the professional competence of thf faculty may not be, a sufficient 
indicator of effective instruction. Differ^n£^ instructional 
, formats m^^y be successes or fal^lures depending on the ^particular 
^ua-l it ies. 0 f the faculty, the students, and. the material to be 
learned.' And observers will inevitably differ in their evaluations 

of . student achievements, instructional ef feotiveness, and curric- 

f ^. -■= ■ . ■ . . 

ul.ar* appropria^b«nese . They can only be brought to ^ctfaptable 



? 
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leyeli of agreement, if they are clear about the purposes of the 
evaluation and the natore of the , standards they are to apply. 
^In addition to differfences in observers, separate components of. 
instructional effectiveness may not be consistent. Student 
achievement may be high, for example') despite a mediocre faculty 
or a poorly structured curriculum if only one of those weaknesses 
is/present and other conditions are favorable. 

In short ,J evaluativ^e procedures ^that seem most pS^sible may 
produce misleading results. A systematic process for exOTinfhg^how 
apparently sensible procedures actually function, in terms of con-'^ 
sifetency ,and empirically suppprted interpretatiohs , ate necessary 
to sound evaluations. As with the recognition criteria and pro- 
cedures, the complex ity o f accreditation is such that evaluation 
of the, reliability and validity of standards can probably be 
carried out most productively if it is done in 1 imited pieces that 
are. cumul ated over time. A process is therefore recommended lhat 

V 

t -J ■ ' \ ■ ^ 

IS modest in scope, comparatively intense within that scope, and 
continuous rather than intermittent// / 

. ; / ■ ; ^ * 

SUMMARY EVALUATION'OF REaOGNITION^fllTERIA AND NdCEDURES ' 



Several aspects of the recognition process stand out clearly 
in the body of information that^' has-been presented. Representa- 
tives of a broad^ apectrum of t^e constituencies of accrediting 
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consider almost all the criteria to-be important and see no addi- 
tional issues that should be included. Most of the criteria refer 
jLQ_exp.liait^_.neadiay-obs&rvBd-^spBets-of^ 

procedures that make sound Bducstional evaluation probable. In... 
scope and contant, the criteria serve their purpose well. 

The application of the criteria is consistent in making the 
same kind of broad distinctions amen| agencies in any given year 
and* from year to year. The decisions reached reflect distinctions 
that experts' agree are appropriate and important, and the inde- 
pendent JudgmBnts of experts largely agree with actual recognition 
decisions. ' The recognit ion" process is therefore functioning 
effectively, resulting in the intended kinds of discriminations 
among^ accrediting bodies. Further, an appreciable^proportion of 
accrediting agency directors attribute improvement in their opera- 
ti6ns to the 'recognition pfocess. 



Nevertheless, the priaent study did: show some weaknesses. 
Views of the impor t ance ,o f aaveral criteria 'were ^mixed; about as 
Tiany agency directors and oth^er experts judged these criteria 
jn important as important. One of them--the criterion requiring 
Dublic representation on governing bodies—was among .the criteria 
ihat were particularly influential in discriminating recognized 
igencies from- th'ose denied recognition. The validity of the 
jroeess suffered to the extent that the Influence of that^criteriOn 
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on th© decisiDns was out of proportion to its importance, Correc- 

^tive action should not^ however, take the Por*7i of a formalized 

system of weights to be attached to the cfitsria in proportion to 

their importance, but should continue to rely on the Advisory 

Committee's judgment in each particular decision. A limited but 

continuing program, of evaluation of the criteria would provide the 

Cqmmittee with appropriate information on which to -base their 
f 

necessarily compleK Judgments* The present study, for example, may 
lead the Committee to reevaluate the importance it has placed on 
public representation* 

* * .. 

A more serious weakness is the difficulty of evaluating. 

the merit or effectiveness of an agency's activities, About'a 

third of the criteria require Judgments o f ef fectiveness'--the 

/ ' , ^ 

adequacy of the agency's staff and financial resources, the fair- 

.■ . ' j""' 

ness of its decisional .the integrity of its gct-icns. These 

criteria are all Judged to be high in importance but difficult to 
observe. Elaborating their definitions and develofiihg a wider 
range of &bse;*vable qualities to serve as indicators of an agency's 
success in meeting them would go farther than any other action to 
improve the validity of the recognition system* Recognition then 
would imply not only that an agency had the organizational struc- 
ture and prescribed procedures in place to carry out educational 
evaluations tffectivelyp but that that effectiveness, had been 
.demonstrated* , ; c % - ^ = 
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APPENDIX A 

Criteria for Nationally Recognized Accrediting Agencies 

(a) FuftcCionai Aspects 

(1) Scope of operations^ ' ' , 

(i) The agency or association Is national or regional in its scope of 
operationa. . f 

(it) The agency or association clearly .afines in Its charter, by-laws 
or accrediting standards the scope of : its activities, including 
the geographical area and the types and levels of institutions or 
programs covered . ■ 



(2) Organization: , \ 

(1) The agency or association has the adrunistrative personnel and 

procedures to carry ^out- its operations in a timely and effective 
^^nner< 



(11) The agency or _assQclation defines its fiscal needs, manages Its 
expenditures, 'and has adequate financial resources to carry out ^ 
its. operations, as shown by an eKternally audited financial 
^ statement. " - . ^ 

(ill) The agency's or association ^ p fees, if any, for the accreditation 
process do not exceed the reasonable cost of sustaining and 
improving the process. 

. (iv) The agency or association uses competent and knowledgeable 

persons, qualified by experience and training^ and selects. sLch 
persons in accordance with nondiscriminatory practices* (A) to \ 
participate on visiting evaluation, teAms; (B) to engage in con- 
, sultative services for the evaluation and^'accreditation process; 
and (C) to serve on policy and decision-^raaklng bodies, 

(v) The agency or association includes on each visiting evaluation 
team at least one person who is not a/member ot its policy or 
decision-making body or its adralniatratlve staff. 

(3) Procedures: 

' ,„ ' ' 

= (i) The agency, or association maintains clear definitions of each 

level of accreditation status and has clearly written procedures 

# for'grantlng, denying, reaffirming, revoking, and 'reinstating 

such accredited statuses* 

(ii) The agency or association, if it has developed a preaccreditation 
status, provides for the application of criteria and procedures 
related In an appropriate manner to those, employed for 
accf editation, , , 
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(til) The agency or aisociation tjequlres, as an integral p6.rt of ics 
accrediting process, institutional or/ program .self-analysis and 
an on-site review by a visiting team, 

'(A) The self . analys'Ls shall be a qualitative assessment^ of the 
' = strengths .and limitations of the Instituticn or program, 
i«ncluding the achievement" of institutional or program 
/ objectives^ and should involve a represetitative portion of 

^_ _ che^nsti_^^^ teaching faculty, 

. scudents, governing - body , anT~dther^ppropf ii^e const i^u^ 

encles * * . ^ ' 

: (B) The agency or association provides written' and consultative 
guidance to the institution or^ program and Co the vlalcing 
team, _ ' / ■ 

(b) Rasponslb ility^-= ^— - - . 



(1) Its accreditation in the fi^ld in which it operates serves clearly 
-Identified needs J, as*followss ' , . 

(i)^The agency's or association' s ' accreditation program takes into 
= ; account the rights^ responsibilities, and interests of students, 

% the general publics chfr academic , prof en^ional, qr pccupational- 

fields involved, atid Institutions* , ^ . \ ? 

!■■■ 

(ii) The agency's or assbciacion's purposes and objectives are clear- 
ly defined in its charter^ by=laws, or accrediting standards. 

(2) It is responsible to the public Interest, in thati 

"(i) The agency or. association includes representatives of the public 
in Ids policy and decision-making bodies, or In an advisory or 
consultative capacity that assures attent.lnn by the policy^and 
decision-maklng bodies. . , ' ' 

(ii) The agency or association publishes or otherwise makes publicly 
available t \ . 

(A) The standards by which institutions or programs are evalu- 
ated; ^ 

(B) The procedures utilized in arriving at decisions regarding 
the accreditatian status ^df an. institution or program; ' 

(C) The current accreditation status cf Ihstituticns or programs 
and the date of the next currently scheduled review or 
reconsideration of accreditation^ \ 

(D) The names and affiliations of members of its policy and 
decision-making bodies, u^^d the name(s) of .t^ts principal 
administrative personneli , 

(E) A description of the oTOershlpi control and type of legal 
organization of the. agency or association* . ^ 

(lit) Thrf- agency or association provides advance rtotice of proposed or 
revised standards to ^allvpersoni. Institutions , and organisations 
.signlfleantly affected, by Its accrediting process, and provides /r 
such persons, institutions and organizations adequate opportunity 
to cdmraenfc on such standards prior to theit adoption. 
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(iv) The agency or association has written procedures for the review 
of complainta pertaining to Institutional or\proiram quality, as 
these relate to, the agency's standards and ^temonstratea chat suc'i 
•procedures are adequace to provldr timely tiveatment of such com- 
plaints in a manner that is fair and equitabie"S-,o the complainant 
and to the institution or program. ' . 

(3) It assures due process in its accreJiclng procedures, as demonstrated ' 

in par r by : _ ^ ■ 



a) 



ii) Affording initial evaluation "of the institutions or programs only 
when the chief executive officer of thij institution applies for 
accredication of the Institution or of its programs;. 

(il) Providing for adequate discussion during an on-site visit between 
Che visicing team and the faculty, administrative staff, stu- 
dents , and t>ther appropriate persons; 

(iii) Furnishing, as a result- of ah evaluation vislc, a written report 
to the institution or program commenting on areas of strengths, 
areas needing improvement and, when approprlaCe, suggesting means 
of improvement and Including specific areas, if any, where the 
institution or program may not be in compliance with the agency's 
standards; 

(iv) Providing the chief executive officer of the institution or pro- 
gram with an opportunity to coninent upon the written report and 
to file supplemental materials pertinent to the facts and conclu- 
sions in the written report of the visiting team before the 
accrediting agency or association takes action on the report: ' 

(v) Evaluating, when appropriate, the j'eport of the visiting team in 
the presence of a member' of the team, preferably- the chairman; 

(vi) Pro%^iding for the. withdrawal of accreditation only for cause, 

after review, or when the institucion or" program does not permit 
Veavaluation, after due notice; 

.(vii) Providing the chief axe-utive officer of .the institution with a 

.specific Btatement..of . easons for any adverse accrediting awlon, 
and notice of the right to appeal such action; ■ 

(viii) Establishing and implementing published rules of procedures 
regarding appeals which will provide for: 
, (A) No change in the Accreditation status of. the institution or 
program peniing disposition pf an appeal; 

(B) Right to a hearing before the appeal body; ^ ? 

(C) Supplying the chief executive of ficer of the Instit^ution with. 
, a written decision of the appeaj. body, Inciudlng a statement 

of "specifics. 

/ ' - = . . ' ■ . ■ ' ':■ 

It has: demonstrated capability and willlngnei^s to foster ethical prac- 
tices among the institutions or programs which it accredits. Including 
equitable student tuition refunds and nondiscriminatory practices \n ^ 



admLisions and employment, 

(3) It maiatains a program of evaluation of its educational standards 
designed to assess their validity ^and reliability* 

(6) It secures sufficient qualitative information regarding Che institu- 
tion or program which shows an on^going program of evaluff^tion of out- 
puts consistent with the educational goaLs of th% institution or 
program, . 

(7) . It encourages experimental and innovative programs to the exuent that 

these are conceived and implemented In a mannar which ensures the 
quality and integrity of the institution or program. 

■ ^ _ ^ ~ 

(8} It accredits only those Institutions or programs which meet' its pub-- 
lished standards, and demonstrates that its standards, pbllcles, and 
prccedures are fairly applied and that its evaluations are conducted 
and decisions rendered under conditions that assure an impartial and 
objective judgment. . = 

(9) It reevaluates at reasonable intervals institutions or progTams which. 
It has accredited, , 

(10) It requires that any reference to its accreditation of accredited 

institutions and programs clearly specifies the areas and levels for 
which accreditation has been received. 



(c) Reliability 



/ 



(1) Acceptance throughout the United States of its policies, evaluation 
■methods, and decisions by educators, educattonal institutd/ons ^ licen= 
sing bodies f practitioners, and employers; ^ ^ . 

(2) Regular review of its standards, policies and proceduress in order 
chat the evaluative process shall support constructive. analysiss em^ 
.phasize factors of critical , importances and reflect the educational 

: and training needs of the^ student; 

(3) ahoc less than two years* eKperience as an accrediting agency or ' 
association; = 

(4) Reflection in l^he composition, of its policy and dacision=making 
.bodies of the corranunit)^ of Interests directly affected by the scope. 

of its accreditation. ' . \ 

(J) Autonomy . ' - ') 

^ , ' . " ' . ' ■ / \ 

(1) It performs no functibn that would be 'inconsistent with the formatloti, 
of an inde^pendent judgment of the quality pf an educational program 
or Institution! ' = ^ 

(2) It provides In its operating procedures again^£ conflict of interest ; 
in the rendering of its judgments and decisions. 
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IMPORTANCE OF ACCREOITiNG AGENCY CWIRACTERISTICS' 



HOW IMPORTANT ARE, THE FOLLOWING CHARACTERT^ICS TO YOUR 
■AGENCY'S EFFECTiVENESg IN HONITORI.NG EDUCATIONAL QUALITY? 

Pliasf eircli ont number aftir 



SCO?l A:Q ORaANlZATIOK: THE CLARITY OF THE ACENCT'S 
..PU^PC^ES, THE EFFECTIVENESS OF ITS ORCANlZATIONAL 
STRUCTrREj A^m THE ADEQUACY OF ITS RESOURCES, 

020 Tii^ ty^^o and livali of insticutions or prograss accredited 
are clearly dtfined In publishtd doeurnents. 

021 The agency's purposii and/or statutory authority ary clearly 
SLatwd and made publicly availablep 

005 The aE^wncy is tht only accriditlng or approval body for itB 
purpo-d^ in its ragion^ . 

070 The agency dtfinei Its own fiscal needs and .controli its 
own bud gee, 

0?1 Exierr.ji fiscal audits of thM agency ara perforaed regularly 
and fin.ihcial statemi^nts are published* 

072 Tht j|;wncy haa the financial riSOurcis to carry out its 
jatiVille^ effgcclvely. ^ - 

100 The agency's fees do not excted the reasonable cost of 
sustaining^ and irsprovini the accreditation process. 

030 The staft is. able to carry out the agency* s activities with'^ 
4 cart* and wichout undue delay* 

102 The agyncy*s fees are set according to a predetermined 
schfe'dulodj such as one based on enrollment figures. 

^ ■ 

■002 ^ The scope of the agency's activities is neither too broad 
nor too limited in relation to the need for appropriate 
accrediting or approval services, ^ ' ■ 




13 



10 



11 



27 



Per centaees 
(N 92) 

1 3 



IB 23 57 



1 7 iS 21 56 

7 11 1 1 8 48 

3 ^ 6 10 23 S3 

16 U 32 

i 16- 72 

10 23 40^ 



3 8 11 



1 4 



. 1 

4 8 8 



9 19 70 



9 2^ 17 



2 2 14 12 25 43 
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IMPORTAHCI OF ACCRfOITlNa AGENCY CHAftACTERISTICS 



* -CW I^PORTAHT ARE THE FOLLOW tNG CHARACTERISTICS TO YOUR 
I AaiNCY'S 'EFFECTIVENESS IN 'CNITORING lOUCATlQNAL QUALiTY? 

Piiisa cirtile ont nymfaer after 



631 
ni4 



pROCED^ESj THE SFECirtCm OF THE 
AC£N:CY*S FROCEDimES THEIR PERTINENCE TO ETS 

Insc lEuC ions ot prograsa are cliarl/ Informed ot the 
natuf^ and aopm of chair aacridlcatlon icacus^ 

Th^ Jet inlclons'of each Uvtl of acergdliiaElQn or 
approval gfancad ^re claar iind wall^dlf f@rinc l.acad* 

Froe^dur€9 for gran^lnga denying, '^IchdrawLng, or 
reln^c jcing ac^r^dLcsd ^^eatus ae all levels art 
clearly described* 

tnaEtcueLons or programs are reg^l^arly reviewed ar^ 
rtja^onable Lncervals* 




S 16 76 



2 I 10 19' 59 



2 4 1 IS 72 



2 3 21 73 



393. 



:35 



512 



370 



When circy@acanees, warrant LC| Lnytltu^lons or pfpgramj 
are feevaluace^^^ Inceevals ^horeer than the normal 



]4Q ^he decli ion-'taakir.g process used to dt 



of an in^cicurion or progras Is published 



The flrsc evaluation of an Insclcuttor 
iniclated in response to an appllcatic 
e.^^cutive ocficer of che LniCtcucign 



tannine the status 



or program Is 
n from th^-chief^ 



ihij agertty 



rgd itat Ion or approval by thij agenjfcy a^dures Chat 

are occupat ionally or profeys lonally {{uaLLfiud 
1 jr jppraprtace anspioysynt. ^ 

The agencyVs procedures allow prograss or inHtifutlons 
rreedom to use a variety of ^dycatLanttl meehodsv^ 

TftatLwutloni or programs are required tO" dubiatt regular 
r-jj>ort^ Indlaacing a continuing profras of self- 
^valuacion. 



15 



2 1 3 ,25 66 



a 17 20 32 



4 1 7 7 3" 13 36 



2 4 2 4 12 17 43 



3 13 25 . 38 



3 1 4 6 12 20 



210 



Procedurts are provided for the aova trota preaccredltation 
to full accredieae ioQ/ 



31 



3 8 10 11 34 



The agency, provides bo^h Corral and informal consultations 
to i>r^accredieed Institutions eo aisist ches in attaining 
full accreditat ionp 



13 



1 2 6 17 24 37 
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IMPORTANCI Qf ACCREDITING AGINCY CHARACTIillSTICS 



HOW IHPORTANT mi THf FOLLOWING CHARACTERISTICS TO, YOU& 
AGINCt'S ifFiCTIVENiSS I H MONITOR INQ fOUCATIONAL Q^LITY? 

each ititf^nent. ' . 



EVALlArrE PRoCtDURESi^^' THE CO^^RtHLNS IVrSESS AND - 
CARn WlT>I.Wli!nH INi^lTUTIdNS OR FROCBAMS ^ 

A suH-ituJy anJ Dn-slt,e rtivl^w ory requi^^ that 

provide for the qualitative asfeesissnE of 1W 

InSE Itution ■ i of program's strengths and wtaknesies. 



121 



261 . 



^121 



6D3 



59: 

ii2 



Written Insfructloni,^ guidance » and coniultatlon 
are provided for the ^elf^study and on=slte visits. 

Each insclcuElon or program ii givsn a copy of the 
visiiini c@am'§ report, whigh indicates strengths, 
w^.ikntiHBes ^ and rtf ommendatlons for improvement* 

Procedures and/or cfiteria Ifof the seltcsion of 
visiting team members are clearl^f stlattd. 



ins 

fii^si^iM or rgject particular yi^ 



Atytior.t or programi rs be ^valu^ted say 

ng^ team 



VibitiRg teams are conc4^ed .with in^iiructloni 
FifS^arees, management , =and si^dent services; all four 
^Tti^st a fa cove fed, } ' \ ' " , 1 



Vimclng teams engage in disgussio 
jiJrj iLU raxors , and students. 



T|t.o ^4ency tenspei^s its evaruative ro^^io as not to^ 
«*hJ.i;i^cr che repueatisn or the ^accrdSitat ion or ' 
jfS^rovai ^la^u^ ,of its inst itu^ions /br ptogr^s. 

•D. y . ■ . \ . . > ■ 

Training of; vi^t/ng teasis and Bonitor.'lng ithelr 
pt'f i prmane^ assures consjL^t enf applicat ion of 

The rtftulred qualif icat iona. and selection procedures 
iot th& asency''s polic^f and d^ciiion^iaaklni bbdies 
are prtiscribed^ 



V 



ith faculty;, 

■I 




I 

' NA- 



10 



16 



1 I I i I ! 1 
i I.I I i I I 
1 2 3 4 S 6 7 



2 7 89 



2 5 18 73 



1 . 17^ 79 



1 2 5 9 26 19 36 



5 10 U U - 11 27 



3 1 I 1 5 24 65 

I 2 i4 a: 



16 6 9 9 7 17 21 



2 ♦ 1 1 1 7 16 33 39 



2 1 2 11 17 21 42 
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iHPORTANGE OF ACCRE0ITIS6 AGINCf CHARACTER I ST ICS ' 



HdW INPORTANT ARE THE FOLLOWIHG CHARACTERISTrCS TO YOUa, 
AGEVCY*S EFFICTiVENESS IN MCNITORiHa EOUCATrOHAL QUALltY? 

Pltasi cifcli ant Aumfeer ifttr , 



iTASOAHDS? THE CLAHtlY AND ^\PPROPRI ATEJfESS OP 
THE nitNCY 'i STa^'DaHDS AND nil CAAI WITH W>iICH 
IT DCVELOFS AHQl MAiSTAlNi THEM, 



^50 



571 



S5^ 



HI 



ill 



356 



332 



Tha igiftcy .i\falya§is xhe valldtEy and feLlabtlliy of 

the jgincy scidleors coat inuLng' cgmplIiAnce wLch 
^)CJtiJa^d3 through rogular rapor^H i»u6mitt%d by 
in^e L^^uc itdni Qr progranm* 

v}^ j^^ncy ragularly fivlaws Ua A^andAtdMg pQLletai» 
ind^pro^^durts by tnvlclng iuggiigioni frqs various 



groups havf had an oppor^unLc^ to raacE co cha 



Art aaciv^ program imp rove Jccredi cae-Lafi standards | 
□ r evaluative procedyrts (t.g;'^ dCudi^Sg csnfereneaSi / 
w^rkjhops) is cacrisd auE* ^ ' h 

riiw i^'oncy'i ^C iiidafdH .icq elt^ar *iid Jut.i i U*y , wnuu^Oi 
iCf 4?^TUfy tair and TiiHabiu Eryaim^itt td Liibl iluLiuus 
ar pfv^gra^i 

The igency sonlfors edueacloaal ssandards tio kiap , 
shea constiesnc" vteh occupational trends* 

tr> ^L^ucLuni or program^ are yvaludt^d with reapacc ' 
to tliuLr o\m purposes rather' chan in comgarlaon with . 




. 2 



2 6 S i7 24 46 

^3 3 5 * i? 24 ^ 

1 2 3 15 33 45'^ 

/ 

1 r 's |s 19 64 

15 3 'l3. 33 35 

2 l 10 23 64 
5 I 16. 24 ,49 

3 ' S 12 16 55 



^ Hie .i^tncy publlHhus the standards by which in^titutigna 
07 prv'^fams aca uvaluatwdi 



SSS Standards are valtdaEed ag^lnsc .the perfor^nce 
griduates of accredited^ Inst Itut tons j giich 33 In 
lleensura fijcaslnatlona* 

$42 Inrorniatton regardlni ytandardi and guidelines for 
, cdnduct of site vtslci is escchanged with other 
/ accrediting agenelea. 



\ 



20 



9 * 



5 10 as/ 



a a 10 19 ;M9 /18%, 



a % 9 ^2i;>Li^i20^|^p 



IMPORTANCf or ACCR£DITIf«3 AGENCY CHAftACTERISTICS 



IM?yRTAf»T ARE THE FOLIOWINS CHARACTERISTICS TO YOUR 

tfnatmm in MONiTORiHa EoucATioHAi quality? 

jse circle ent nymber afttr / ^' 



S^IV£:it:3S^ WE ipSlTIVITY Of THE AC€!iaf 
T.^ Uil A,ND DESl^S Or THE CROUPS IT ' AFF ICTS , 

A-.:- lis RE^ISfSi TO RpSFOHD TO TJ!Pt. ' ' ' 

T*^c yti^^^y hap pu&lie rep/esentaElves Qn l£i peliey^ ' 

All fel^ni^icani, Inr^rtsttd greyp^^re-f^^ffitneed on 

r^rriinc sec red U it ion icagyfeS of lni&i£UElon^ of = 
fjr;*;r,iss ^nd d^t^§ ©{ the tiext sahedultd review are - 

SolifU4Eian oi thm viiwi of s^ydtECf is psr^ ef th# 

K r./!n,ill0R ei'. 4?pr*ii»^l process. 

* " . Qt aprfspfiJiifi ^cAde^le^ oceyp^iionali or 

^u , Sftmb^fgt if€ tiv^n opp©rtyfilEi#s to mak# xh^lr 
. = t -^^i^ c& £h*f ^^^acy. ' . . ^. ........ 



rf H^^^^^ or insrifucions In its field ©r jufisdleLltift, 
* .i..f. ;.t.i:uin ^jf jp^roval frost Ehe aggncy. 



^ ^' ^^^^r^^^ ^' "-^'^ ^geney'a owntrfihipi conrrel* And 



.tACion and tnnov-'ton are tncouragod* 




t 
I 

NA 



?it'I^wis JLfh p^ff^onn^l in ae^SdfdarigO with 



\ 



1. »^ 


1 




1 ' 


5' 6 


T 







I t 



'6 2 10 13 14 17 '31 

■ • , ■■=■ 

'3 2.^, 10 9 18 14 37, 

'A ■ 

2 3 a 8 14 '20 44 

1 .2 2 14 2^ 47 V 

I . 2fe 2 .7 1? . 69. V 

■ . . - •• } ^ - - 

1 - '9 31 59 • 

1 2 11 11 24 so ^ ^ 

4 3 9 10 12 14 41 J' 

4 2 10 8 10 19 4S V 

4 11 4 20 29 40 

2 . 6 15 11 r20' 44 / 
4 1 6 9 10 17 53/ 



IMPORTANCE OF ACCREDITIHi AGINCY C^RACTIRISTICS 



HOW iMPCRTANT ARE THt fJOLLQWING CHARACTERISTICS TO YOUR 
\ AGENCY- S EFFECTIVENESS IN ?^NITOR,lNG EOUCATtONAL QUALITYI 

'\p1%ast^;eTrSle en^ number afttr * 
= ^idch ■siaiimtnt/', w • / .a ' ^ . V " 

: ■ * . / * . _ I 

, ' ^STRUCTy^D X^D CODlFtlD to AVOID^^^^^ . ' . 

- . CAfRECtbyi; ^lOHS* ^ ^ /"^-^^ ^" - . 

• • ■ . . ^ - ^X" V... / - 'V - ^^ . 

- , * .^^f 4^p4ind EQ Eha vt^iilng" etas i^porc« . ^ ^ " 
.* - ■ ^ ' ' : . . ^ 

450 V 4t I eA^^^ne number ef Eht_ vtsle tng Eia9\ Is^faqulred 
. , CO preiehc #e she gvaluattoft of ihm viilEiaf taam 
' . H . report. / ' ' 

460 Accr^idicacLon is wlchdrawn only afC€r due pr^^tesi p 



410 



500 



414 



f^r. cmy.il, and with dua nocli^i^; 



Th« agengy repdrEa che spe^elf ic rtaioai fqr sdvtrse 
cht^f a 



d#cLiion9 CO che chl^f ax#gutivt offletr^gf khm 
lngcitucio|i or pro^ras^and offtrs an opporcunicy £o 
3Dpaal. 

i . ^ ■ - /■■' - ^ ■ , 

Advar^e dee Is ioni/i re followed by a Jiearlng at which 
-raprpdahtattvesyOf Ehe affietid iniclcucion or 
program may-appr4ar. - 

Appaai procedures include safeguards jgainit biair 
a,g,, no on#- involved in Che originals decliton may 
ierva on.^^he^appeal cosmlCEift. ■ ' 

Tite agena^; hasN pubilshad rules of procadura regarding 
appeals* ' \- ^ ^ ^ 

^ ^ \ ' ^ 

^0 An inst icic'lon' s\or prograB* s aeGredlcaclon icacus 
cannoc ba changed by cha agthc^ while an appaal is 
pending. ; ^ ^ ! 

. ■■ - I . ■ ^ ^ 

BIO \*iaElficacion of specific rtaions for ^ an, adverse aect^n 
- ■■ on an appeal ts required* - ' ^ 

5^0 The agency's pubilshad procaduras ara fairly appliad; . ' 
l§s actloni are consistant with its procadurei. 




14 



io 



I . 17-^ 



7 15^73 



6 12 3S 



3 19 76 



'2i 74 



9- 18^ 



7 16 51 



'§ U V 70, 



11 16 61 



9 17 67 



4 12 

m 



80 
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IMPORTANCE OF ACCREDITINQ AGENCy CHARACTEftlsf LCS 



•HOW IMPORTANT ARE THE FOLLOWING CHAWCTERiSTICS fTO YOUR 
AGINCY-»S;EFF£CTIVENESS IN MONITORING EDUCATIONAy QUALITy? 

Pltiaii circli tfnt numbif after ' / | ! i 



720 



AHENCV INTySRITYi .THE DECREE OF.TRUIT EH6ENDERED 
lY THE AGENCY THROUCH IT! RELIABILITY* AUTONOKY , 
AND eONCE^ FOR ETIUCAL PRACTICE,^ U 

Tht agency is noc eonscfal^ffd. In its acckoni or 
decisions by tita go a parent or yobrell^a: organi^atllon. 



The agency *i auchoriEy is yEjoadly refogniied by 
educational iniciEucioni, appropriac| iicgnsihg 
agijnei^s/and p,Coltssi§nal 6r occupacional' fields, 
- ■ ^ , ^ . . ■ 1, : r ' / 

5^5 The agency denies candidacy and accradlEation' to 

inscltuEions or programs 1e finds eo be ; uheEhical In 
. conducE * . ■ . . / ; 

520 agency definti, fbsters, and sonitoti ethical 

practices by" InstituEions or prograas viEh regard to 
such aciiviEiesai recrulEing and. adverfci^^ 

Tip; . TIjw agancy perfbrms no function EhaE interferes wiEh 
' - f thy jti^ercise of indipandent; judgmenE abo^t the 
educatlanal^qualiEy of an instltytioh* 



190 



513 

m 

526 
211 

342 



A written policy oh^"s«4^of operating procedures clearly 
guards against cenfllctr of intereit/ 

■■■■ ^ ■ ■ . ' ' ^ 

Xii^-^^-i^^^^^v^^^^^^ at^^ leas E , one peri on who is not i 
tiember of the aienc^^'s pollcy-ffl4kln| b^^dV or administra* 



tlve staff- 



743 Members of the deeiiion-oaking body do/ not partlclpatff 
in diicussioni or d eTt § i ons-_, s f Jfe ct in g i n a t Itu 6 1 ens ^ wi t h7" 
which they are' affiliated. 



Writ ton procedurfe^b clearly osaure that only gualifigd 
inktitUEioni or programs art accredited or approved. 

The agency^fl decisions reflect fair and consistent 
application of its standards. r 

The agency is recogniied by Other accrediting^ 
agencies or by the Couheil on Postsecondary' Accredit 

' tai ion^^ ' r " ' ' 

Monitoring of institution or prograa practices in 
advertisings rfcruitlngs student aid, or .other areas 
is detailed enough to minimise abuses^ 

Concerns for institutional or program performance go 
beyond ^ueltions of educational quality into issues 
such al .ethical/practice and financial atabilifcy. 

The agency ensurei that institutions maintain sound 
educaE lonal^/centrQl of off ^campus or contracted programs* 

The agency rtquifts that applicants be given' specif led ^ 
kinds of inrorm^tlsn. about the institution or program, 
such 4^' c^aplatlon rates or employment rates^f graduatei* 




5 



10 



12 



11 




14 



20 



14 60 



8 li\l8 54 



7 25 58 



1 S 17 18 41 



# 18 66-. 



32 47^ 
/ 



■6 21 54 



7 14 69 



3 22 72 



4 17 77 



6 14 



8 22 20 .30 

7 10 33 31 

1 8 27 46 

9 . 9 21 21 
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= The tlghcy i^eelfle igeney eharacctrlsties you have just J yd lei for abtolutt 
tBpgftanee were grouped incs sivea broad areas of eeneern.^^'^New we would 
like you to . rank those seyen/general eonGerns,' ?eit3Eed beloWi "^tflto their, 
comparaclye order of Imporeanee In aisurlag thaE, your agtriey wlli\fun€Elofi 
ef fietively,4i a feliable ayehorlty with reipect to the iduaaelenai> quali'ty 
of the prograM o? Institutions with whleh it is concerntd. This will ' 
provide Informati'on about the broad coneentts of aegredltatloQg whleh may be 
Imperfeetly represented by the speoifle statements, 

The ranklmg proQess Is easiest If doae la igroups. First pick the two most 
Important leneral areas of concern as applied to your ageneyi then order 
them and place a "1" and "2" as appropriate la the space beside each 
itatiment. Then pick, the two least Impprtant concerns and mark thtn "6" 
and "7"i recognizing that the^anklngs are relative, and the lowest ranked 
may itlll be^quite^^important/ Finally* rank the remaining three concerns 
from /»3" to "5". 

If an entire area of conptrtt is not applicabie to your aieney, place It in 
one of Ehe Middle caCigerlesj with "hX" noted beside It racher than a rank. 
The two least important concerns should still \e ranked "S'* and "7". 



% I'i n's & i'l 



SCOPE ^SD ORCAHIZATIOH: The clarity of the 
agfncy's purpoies, the tf fectlveness of Its 
orianlzational structurei and the adequacy 

of its resources » . 8*0 13*7 ^ 



ACCREDITIHG FROCEDimEiL Jhe ipiciflclty of 
che agency's procedures and"" their pertinence 

to Its purposes. ^ 6,8 ^ 20,4 



EVALUATIVE PROCEDURii: The comprthenslveneis 

■and care with which Initltutloni' or programs ^ ^ ' ' \^ ' 

are evaluated, 27,3 55.7 



STANDARBSi The clarity and approprlatenf ss 

o£ the agency's standards and the care with ' 
which It develops and malhtains them, " 39,8 72,8^ 



RliPOHSlVEpSi: The seniltlvlty of the , 

agincy to the needi and desires of the groups . 

It affictSt and its readiness to^eipon^ co 

them*^?'^' ; ^ , ^ . 1,1 7,f 



' pUl PROCESS I The care with which procedures , ^ 

are structured and codified to avoid unfair 
' ^ or, capricious actions* 1*1 

^ AGENCY IKTEGRITYr The degree of trust en-. - 
^ genderid by the agency through Its rellabll^v^^ . 

ttyt autonomyp and concern for ethical r ^ 

- ' - = p r ac t fd e * "~ \^ - 17,0 

- ' ' ' ^ ■ _ ' 

Thank you for your Jud^ents, They will help evaluate che current OE 

critirla for recognition* As noted earlier, you will be sent the results 




APPENDIX C 'X 



Mean Ran,ks of Comparative Iffiportanca 
Stacemenc 

The agency employs competent , q^allf led prople as 
participants on visiStog teams, - 

The agency's decisions reflect fair and consistent 
application of ;lti standards* 

A self-study and on-site review are required which 
provide for the qualitative assessment of a program^ 
strengths arid weaknesses* 

Some doubt eklsts as to whether the agency's 
procedures assure ir^^rtlal and objective evaluation 
and decisions. :. 

' The agency publishes the sWndards by which programs 
are evaluated. . N 

Prdc^rea for granting, denx^ng, 'withdrawing, orv ; 
reljistatlng accredited status Vt all levels are/, 
clearly described, ' ' 

The standards by ^whlch the agency evaluates 
Institutions have no^ been defnonstrated to be valid 
.or reliable. , , ? ' ^ ^ ^ ; 

The agency dbes not provide sufficient preparation 
or instructions to^ the visiting^ teams* ^ 

* . ' . >^ ' ' . ■ - ■ ' .' . 

The ag^cy's authority Is broadly recognized by 
educational institutions, ; appropi^iate 'licensing ' 
agencieap and ^rofesilorial fields* 

Each institution is given a copy of the visiting 
taam'i'reportp which indicates strengths and weak- 
nesses and makes reconanendations for improvement. 

TheTgency's prpcedures for granting or denying 
accredited status are not clearly stated in its 
publications,^ 

Visiting teams are concerned with iti struct ion, ^ 
resources, management, knd student services; all 
four areas are covfered.; 

Written procedures clearly assure that only% 
qualified programs are^gjoved. 




, * . = Statement , . ^ >Iean Rank^ 

253=. The suggesced structure of the aelf-study does hot ^ ,3*5 

assure k iufflcieatly critical d^\analytlc aRiess-* 
merit of the strengths and waaknesses of the Insti'- / 
. tutions. . ■ ... ' 

128. , Written instructlpnis guidance, and consultation ■ 3.5 

are prdvided for the self-study and on-site viaitsr 



54, Programs are ragularly reyleweu ^t reasonable 3.4 
Intervals. ^ . 

552, The agency regularly reviews its standards, - policies, 3,4 
and procedures by inviting suggestions from various ^ 

' ' interested groups, " 

< ■ ' '■ ' . 

^%0* The agency r^p^rts uhe specific reasons for adverse ' .3*4 

N decisions to thm director df the program and offers ^ 

^ V ' . opportunity to appeal, ' 'v . ■ . / 



?^8, The agency's procadutes "tor withdfawlAg or revoking 3.3 
approyal statuses and/or for ralnstating programs 
are not clear. " "■ 

460. \- Accredltatiofi is t^ithdrawn" only afti^ due process, ' 3,3 
for cause I and with due notletf. ^ f-'-- 

. ' \' ■ ■ ' "^^ ; / \. ^ ■ ^ ■ NC^ 

576* The agency's -procedures for observing a prpg^ 3.2 
evaluation of its outputs are not clear, Jf^ ' 

21, ^ The agency's purpesps are clearly stated and mada 3.1 
publiclyravallabie* " ' ^ U= ^ ' ■ 

129, Tha agency praparas visiting team membars through ; *3.1 
\ workshops! training sessions\ or similar activities. ^ ' 

;382:* An active p^bgram^^to improva approval: or evalui tive 3.1 
. procedures (e.g. ^ studias,: eonfertttces, workshops)" 
. is carriad out* ^ / --.^ ^^^f ' 

41, ^ The dafinitlons of each level of accreditation 3.0 
graated ara plaar and well dlflarentiated. / 

, 6^. A wrlttan policy or set of operating procedures - 3.0 
^ 'clearly guards againat conflicts of. Interest. 

'20. The types and levels of programs ictredited are 3.0 
clearly dafinad in published dqcumants, _ \ - 

488, Tha agency does not. have a defined process , 2,9 

. regarding appeals. < 



3m 



' Statement 

^he dgency perf Drms no function incorisiitant with 
.making independent judgraenta about the educational 
quality of a program^ 

. Prog^ami are evaluated with respect to tkeir Q%m 
purpoiea rather ^han in comparison with others, 

.Thavdirector of the^ evaluated program is given an ^ 
opportunity to respond to the visiting team report. 

The agancy's standardi of ethical practice are not 
clear with regWrd to^ matters such as racruitlng and 
advertising. \ 

The staff is able to carry out the agency'-s 
activities Mt^ cMre and . without undue delay.; ' " 

Adversa decisions ar% followed by a hearing at which 
representatives of the affected prngram may appear* 

Vcltcen proaedures provide for fair and timely 

- treatment-.by * the agency of com 

instltu^orii; / 

^ The^ agency has-pabllphed rules of procedure refardini 
appeals* ' ' 

. ' - I ■ . ■ . ^ . 

- . I ' " ^ . . . ^ 

Standards are; no/t revised until after ^interested 
groups have had /invopport,unitj^ react to the 
proposed revisi0ns. . / ^ ^ , ^ 

'Y^^^^^^S Warns/lnclude at .least one person who la 
not a member ^f the agency's policy-making body or 
adminlstrativeVsta^f i : ; V ; / 

The agency defines,, fosters, arid monitors ethlcai- 
pwctices by institutions with regard to iuch 
adtivitles as recruiting and advertising. 

Programs are required to submit regular reports 
indicating a continuing pragram of sel^-evaluatlon* 

The current accreditation statuses of pirdgraras and 
the datep of the nejcC icheduled review ire available 

to theTpublic^ s 

At least one member of the visiting team is required 
to be present at the -evaluation of tlii. visiting team 
report, , ... t.'.- V 
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283. , 

sso. 

490. 

78. 

640. 

410. 

100. " 
71. 
91. , 
IS.. 

102. 



The agency hai publ 
policy-making bodl<i 



Statament 

preienatativea on its 



ExpArimentation .md innovation- by programi are 
encoufaged. 

An institution i.aceredicat. ion status e^^^^ be 
changed by the agency while an appeal is panding. 

■ . = ' . • = = . , ^ 

The ageney'a iourcas of revenue are rtade explicit* 

The agency maintains contact^ with counterpart 
.agencies in other stateif . 



: The first evaluation of an Institution is. Initiated 
In .response to an application;; from the chief ^ '\ . 
executive officer pffihe institution.^ 

The .agency's fees do not axceed thi reasonable cost 
of sustaining and improving the accreditation process. 

External fiscal audits of the agency are perfQr,TOed 

. regularly -and financial statemeints are published, 

'''''' ^ , ■ " ' ' ' ' I - 

Infortaation is too incomplete or ambiguous to 
decermlne the agency/ s financial strength.^ 

Thff, agency Vs 'Junctions overlap with those of at least 
one other agency. . - 

The; agency' 3 fees are jet according to a prkdetermined 
schedule J "such as one ba^ed" on enrollment figures. 

. ^ ^ . \ . '^^ ' '. ■ I 

The organtzafcional structure and staff siie ol the 
agency are published, : ^ 



Mean^ lank' 
2* 2 

..2i2 

2.2 

2.1 
2.1 

2.0 

1.9 



a' 



1-.9 



1.7 



1.6 



1.5 ■ 



::i.4 



Jecause^of the forced sytninatrical nature bfs the responaes from 1 to 5, 
Che overall mean for 215 KaspondBnts was 2^95 with a standard daviation u 



of 



. 75. 



ERIC 
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;appekdix a 



Vhleh of the followini changts have ' - 
V Occurred In your .ageney since 1969? 

were thm cise and origin, of Che 
change f ; ^ r ^ 

: . Flease ^rcle One nusber afcer, every 
r:; tiaceffienc of chMigt and an additional 
^: leccer whenever 1, 2^ or 3 was circled » 



i^andard^ btcame baceer devlned, more 
expIiclL^ 

SEandaTjs becaaia io narrowly defined / 
that they often ^lis laportant educaclonal 

lisues* 



di^CES IK ACCMDITIHG AHp APPROVAL 




Initir^jElong indreaiingly saw our accrediting 
process as legalistic or heavy^hardrd*/ 



-.The agencyybecaae more iyscematlc in /Its 
cdncem for the valid 5, ty and rellabllicy 
of its standards, ^ / 



92 



75 



13 



vThe agency pyt InEo effect s cont4.nulng 
prograa of review dealing with subi tan cive 
changes in, Instiiutioni or prdg^^s, 

. 

The agency's st^ndarft- w^ri i^renjitheiuii* 

In. applying standardi, the tmphasls sliiftad . 
/from the ^encouraoiBtnt ©f iaproveaent £o 
thu identification of weakneiseo> J^r 

'- ■ • ' :'- ■/ r 
IniEltutlons or programs^ became Increasingly 

erlcical of the way standards were being. 

defined and applied. 



33 ' 12 
18 2 

; 

.74 ^ 6 



COMMUNICATION 



9/ ' The a^.ency began publishing Ita pollcfes 
and procedures or clarified their statements, 

10. The agency becase aware of a n/ied for becter 
co^unicaCion.,wlthC federal 'and state agencles» 

11* In foraa't ion sharing with other organ iiat Ions 
through conferenceii hemringit^or workshops 
was increased, : " ^ 



29 15 
33 . 11 



23 



Note 4.. Figures are perctncaies based on 91 reiponsesp 



C>UHGES IN ACCiUDITIHG AND APFRQVAL 



Which of ihe fQllowtni changts hive 
.occurred In four a|ency since J.9if ? 
Whac wi»rt che cime and priglB^f £Sa 
change? 

PI ta i 8 ,.clrclt one number after eyery 
aeaetsenL of change and" M addlclonal 
laccer whenever . I, 2 p of 3 wai circled. 




12* The agency, ^improved iervlce to its field 

- by broadening Ics scopep . , ^ ^ 

• ' & . . . 

13*. The agjency's growth or extension ..of iCQpe 
. ' WMi curtaHed. ^ . 



45 



90 



IL 



14. 



Ths burden ImpQi^id- by OE requlremencs. 
IncroaJ^d* . 



42 



15* The org jhlzac tonal iEructure^^f the aiency 
was changed. 



50 



11 



RESPUNSIVEHESS 



Pfocodures for-d#aIing with eomplaincf 
againic-- Inac liu^ions were imp roved « 

17, The burden of dealing with rtacclona by 
init itut ioni» programi, or .consumers. to 
OE'i riqulremtnts increasedi 

; liv Public repre^QnCac ion was increaied* 

. 19* ^.Tlie/aiincy becama more responslye to ' 
public concefns. 



23 

53 : 

29 
37 



14 

S 
14 



20. The igency .beeame fsbrt conee*fned with the 
ethical pf,acticts of institutions or 
programs* . , 

21*' The InporEance glveru to dtviloplng . 
policy regarding social issyes grew, 

22* The agency 'i procedures .wert adapced^ to 
accommodate Inicitutions or p^rograns that 
depart frotn '■itandardV "educational apprQache 



44 



S3 



43 



10 



10 



1^3. jThe increasing spiclf Iclty of the agency*! 
accreditation process began to stifle 
innovat ion* / 

i4. The agency becaiBe more respoaslva to the 
public deaand for information potentially 
damaging to Institutions or programs* 

2S, The agency became more active In protecting 
Chi public /gainst Institutions or prograai 
' = ' of poor quality,, ; 



93 



67 



47 



310 
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^joeeurrii in yeur igeney slnee Ifii? 
v dhat Werii ch« claa and origin of thg 
^ehahgt? ' * 

Glrdlt one number i fee? evtry 
^i^^careieenE of; change And «n addlelbnal 
rlattir whantvfir I, 2, or 3 was elrcUd* 



C^CES IN ACCREDITING AND APPROVAL 






FROCEDlfRtS. ' . ■/ - , 


0 


1 


2 


t ^ 3; 


a ' 




26, 


"Speelfie levels of a^efidlEatlon were * 
daytloped or elarlfled, * 


39 


17 = 


29 




3 J 


67 


27, 


Tht agancy'f procedures became mnri 
gysEemaE^ or etandirdized,' 


14 ' 


11 ^ 






^ 21 ' 




28. 


^Procedurei for on-alEe viglgs ware ^spTovid, 


11 ■ 


J 


37 ; 


43 


24 " 


76 




Froctdyrei for the self-scudy were Inprovid 
or clarified, . ^ 


4 

2 


9 


48 


^1 


33 


67 y 


30* 


.An appeals procedure was inscleuted or improved. 


^ 26 


19 


40,. 


15 


, 46 ' 


54,;- 


31, 


The accrediting or appreval proceduro was 
depollticiied hy virtue ef the appeals proceii. 


72 


8 




6 


32 / 


63 


32, 


Thf agency 'became more credible because of Its 
^"improved/ procedures, - ^ 


37 


2 


# ■ 
23 


JO 


dU 


aO; 


33, . 


The 'evaluative funcE ion: of the agency became 
clearer to instltucions or programs,, 


32 


3 


23 


40 


19 






The time spent In reviewing each initlcuElon 
or program was Increaped, ■ 


4i 




32 


IS 


20 


80 ^ 


31, 


ConsulcaElori with insEltuElons or programs on 
review procedures was increj^^d. 


34 




30 




.22 

i 


78 - • . 




iPiVlLOPHENT 


K 
V 












36. 


The agency's role and purpose in accreditgElon i 
Of approval were clarified^''* ^ .* , 




.12 ' 


11 


. 28 


30 


: 70 


37* 


The agency has grown. 


-21 


9 


16 


54 


7 


93 


38, 


Planned cMnges came to be iriltiated more quickly. 


47 


1 


35 


17 


25 




39, 


The ielf-icri|tlny required in applying for ^ 
''rtcognltlon-itrtngt.henfd. Chi agency, 


■ 29 




33 


29 


64 


' \ 


6 

AO, 


The agency's fees , for aecreditatlo^n Incjlressed, 


^ 31: : ■ 


... 8, 


."43 


17 


- .17 




41, 


The igency' 8 higher f ees r^began to prevent some 
schools or programs froii seeking accreditaElon. . 




1 


2 


7 


13 


87 . 




Beciiuse of., the agency's connection with eliglblllcy 
Cor federal^fundii accredlEation in effect becaap 
aanda tor y. rather than voluntary. 




11 . 


9 


12 


61 


' 39 : 


43, 


Thi^agency'i functions ware llmljEed or reduged 
by Qi'a raqulrflaenti. . 


92 :^ 


2 


' 3. 


2 


27 
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SUCGfeSTlONS FOR. HODirYlHC THE RECOCNITION 



HOtf,CLOfILY DO THE FOLL'OWTNC STATE>qfNTS 
liFLICT VQUR AQmCi'S CONCEWIS FOR T 
iHPROVlHC THE RICbaNlTION FROCEli? ) 

Pltaie clre la QHii number afcer 



"^RECOf^NDATlQNS FOR ^DITlOHAL CUID#\NC£ 
AND ^ISCLOSUM TO ACCRlDmNG ACEHCIES 




1. The Divisiefi ©f Eilgiblllty and Agansy IvaluaElan (DEAE) 
©f^Ehe U.S» Offiee of Edueacien (01) should publish in 
detail ita prelcdurcs for the ival,uaEiDn of ageneiasi 

2* OEAE should previda mori information on how an agtngy 

will ba afftetad by a particular rac3|nitlon deoision, ... 

J* DEAE should previda aiancits with a eop^y of tha reviaw 

sugary preianttd by Its staff to tha Advisory Co^itcee^ 

4, PrqetJutas and criteria for selactlon of 01 Advisory . 

' . Cop^ittaa mafflbars ^sriould bt published* \ 
1^-^ . " ^ ^ ■ - ' ^ 

3*. The rasponslbilitlas of DEAE and of the Ot Adviiory , 

C con^ittaa should ba publiihed. 

6. DEAl shpuld provlda be&tar dir#ctiens to aianclei f ©r ^ ^ 
^ praparing patitioni, '-^ - ' . ^' ... . 

7, A tifflttabla for iubmittlng m to DEAE ihouid ba 
pfDvidad to agancles. 



8, 



10, 



DEAE should specify the Hindi of evidence it eonslderi 
^aGcaptabia for Beating each crlta^rion* 

The Comnissloner's raport shouid proVida more datailed 
feedbaok on tha raaioni for the racognition deoiiionji 

' . • - r-*:^ ■ ' ...... " ; " ^ ■ 

Many ^rl tar l»a ara vaguaj difficult to interpret, or their 
wording la too lagalistici t^eir Bjaahingi ihiuld be made 
ttora explicit* -^^ ^ 



1% 



18 



13^ 



\ 

11 



Percent*a|ai 
(N - 94) 



IS 5 71 



1 \ 1- S 11 18 33 / 

\ 1 11 16 66 ^ 

; V I 2 J 21 41 

' J .23 13 51 

1 6*8 16 U 35 

2 : i 30 16 37- 

'.. . ^ .1 , - . 

4 1 A ID 19^64"; 

12 4 21 'llj/^B- 

12 1) IL 19 i . 34 
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SUCCESTlONi FOR HOplFYlHn THE U^mHlftm FROCISi 



. HOW CLdSiLY DO THE FOLLQWIHG ITATEKEHtS 
REFLECT YOUR AGENCY 'i CONCE|L»^S FOR 
.iHPROf INC THE, RECOgHITION. FROCEir? 

Fl€ai^' eircis gne nusbtr aCEtr " 



C^>Offi^fTS OH l^ilQillTlES OR INCONSISTENCICS 
IK THE RECOCNITIOH PROCEDUREi' ■ 



-.1 




ii. Hose erietr lis refer to escabilih^d aggnctes ra£h#r chin 

CO Initial applicants. ^ \ 

. " " " "1 - ' ^ . \ 

- ■ . ' .. . ■ ■ '■ ' '■ 

11. Th^' largit pbwefful agancles racitva praftrenilal 

■ ^ \ ' ■ \! ^- . . - ■ ' ^ y 

13*, . The- incirprtiaiion of chf ccicifia by the Advlfofy 

CoiTAlcCie Is' noc Qonilicenc fros one year co^che ne^fc. 

14, DE^VE should be required ce sieiiE t^e artceria ehac . 

a^^nai^s .^ySE sfrer. . " _ '.y . - =■ 

^15* .OLSE'f due process proEedurti , for- acc're^ 

a[>pruvjl iigencies are inadequate. ' ' - T 

l^t Current procedures do noE allew an adequate opp^rtunlcy 
^ ... f©r agisneias ! to respdind to DEA£ concerns^ . . ' ■ - 

17* H «iar Ing .procedures for ageficies should. b@ icrustyred to. 
, entfuro li^at dppropViatt issutjs ara covyrcd* - - ■ * ' 

'i8*f Agunc 1*1^ are not g pr^par^/ 
^ ft'htiic^ls CO i thlrd party ciMlluii>;UH, , . ^ 

49*- Tiw A^Ivisory 'Cosmic i^liuyl!^ ro.iU jj^enay pgi icion^ in 

addit4'"on to DE*\E staff . summjr'iii.*?* , ^ " m 

tQr\ An ac<r,^d icing agency st 3^6; -ojember Should be present at 

the tlfrie of tht agency's reyiew by the Advisory Cpssitcee* 

21* ^' Members of the Adyiiory Coj^B'ittee ihouid have pore 

. : direct «ij€pjirienQe' with- agencies . y ' . , 

ilv The Advisory Csoisit tee should have broader representation 
7"; \ (prof iiilonal i voeationaL^ propriecaryg\ etc. ) ta. improve. 1- 
the quality of Its evaluation i'i^ 7 /. 



43 



48 



39 



26 



4S 



li 



16 



2S 



29 



6 1 
10 n 
1 1 



1 5 



I- 



12 8 21 I 7 



17 a 10 10 12 




14 S 11 8' 17; 

i" ' ' -^^ ■ ■' ■ 

25 ■ 7 13 18 

. i a p \h 

1 10 6 13 ^ :ap 

I 2> 4 6 18 1^ 

2 5 18 IS \ 42 



2 10 21 IS - 24 



2 1 I 7 IS 16 24 
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SUCqiSTI0Nt,vrOR HODlF^lNn THE RECOCNmON 



VHOWiCLOSILY DO THE FOLLOWDIC STATEHtNTS 
: RITLECT YOUR ACfNCY'S CONCE^S FdR 
^;-IMPROVlNC THE RECt^HITldN PROCESS? 

•••• ■ ; ■ • ■ ^ 

: Please el re It on# nuabtr afctr 



SUGCESTIONB rOR CHANCER IN THE 
FUNCTIONS OR ROLE Of OE' 



,23* EEhicsl pracElc€l^"Qf agtaeleiis^ewifVfE desirable, gheuld 
■ 'noc be, A DEAE cdneefQ. . ^ 

2^^-^Eduearl©nal: pulley* ^ueh as g^eaQcing^xpiir^ertEiiEiofi j „. 
; '^""^-sfeould npt be iispltwenEtd through D^E -'^rlEeria* 




21, 



26, 



27. 



31. 



32. 



33* 



34/ 



DEAE ihould hslS open mtecingi to givt ag'enol^i an 
Qppor^nlty io previde InpuE^ on poliey chan^yi. 

Th^f AdvXsDfy 06^11 tee 'should operate ai a forum-for L 
Uiuts ir, Ehfe ovfirall field ©f acereditafeionv- 

:.0E should publish, eemprthenslve lis£s of unr^^^^ ? i : 

as well «i Tecogniied agencies* 

Infor^Elon sbout ehanges In tl£|lbillty bf ^^sEsecdndaty . 
$^f(9pls. fpr federal anfl SEate pr^graffiSj FTC ceaie and 
de^liE orderst rtiErieiiens t&j^ostd by Eht .eoyrcs* and 
accrbdltlpg jor approval aeEtoni'^iheuld be e^re pronpEly ^ , 
■ and- = widely "tijichangad^ ^r'. -^- \, X ': ^ 

OE sliQuld ImitiEuEo a^ pfd^gram dw»8Si*nc*d to It^prove Ehi 
- Effliniriy of ^JtiaEe etdygaE Ton ^laTf with tQBpaai to eligibility 
rcquJremQfiEsy te..gairt; tljolr fulj y^peraElon in enforelng 
aUgibiltt^ cohdltions. • . . vt ^ 

OE shoyld awards* granrsX^r feseai^eh and eonEr^eta fpr ; 
sarylaes eo accredlElng agtficies as one seam by which to 
.refleoc ips ehanglng Idtas of the V^blle . interest , - 

InSEiEMEiofia reetivlng federal fundi ihduld be held > legally 
aeeountable for the honesEy and ayeuraey of the information 
they publish and dlisemt?iaEe aboyE their educstioful 
pfo^r^&i i^d fl^ianeial itatus.. 

itu^ies' ihould h% cdndueted tp eitisate the nusbers and 
binds of»d*gre# and non-degree. gran ting Inst itue lens ; 
Eh^E engage ^ ipecif ied iwlpragtiens* v 

OE should require -E^it agenoies inform InsEiEUEions 4nd 
pfogracis abouE the taosE cosson fdrois of aisreprilienEarion. 

OE ihdu Id permit the i^eognltlen of more Ehan^one ' ■ > 
aecftdlElng ageney In geographic or educattdnsl area* 



47 
J' 

16 
18 



?8 
15 
18 



33. ;> 5 31' 11--. 7 2' ^ 

r 4 18 ;:rt7 2A io:' ^§ 

1 1 6. 2|^^^24^:^V , 



10 5/ 11 6 17 - 13^'^ 
^ ■ . . . * - ' 



2 4 4 11 19 13 24^^ 



;i 2 ? S^Jl iO .if / 
15 4 10 5 2y 11 16 y 



1 8 17 11^ 54 



* 2 7 7.22 U XS 
11 6 11- 18 8 p ^ 
23.^ 5 ■ 21 I "4 8 12 



E'4 



^h;p?:£stsons Fon ^DirvrNCyTHE hcc-^cnition process 



HC, ,.:s:t^^m process 




5?. 



/ 



T.'it? rt»4^rjl lovffrns^oc shpul'd provide r^saarfch and 

lA^U^Mti^njI eU^t^^UiC/ f^r public funds should not- 
>4 ^^nc^fTi^d wt-n n^^^qaul r^iipsnsibUuv ^nrf public 

,A:pTJ&ll.it^ bQdf iot 3Hcncl«i5 denied recognlcis^i; Its 



. -^^Id preserved &y deiegsci^c fteognUsef 
«^ jtn ;iiiiip^ndvnE boJy (eaPA, igf ejtJmpie) which wsuU 
fVfv^ J* buff«:f agjinss political ^f**«i^u?ei. 



5*. ru^^ ftVivW .tn,4*^ncy'i rat ions ihouid' dtl^gjcad 
4 ?riv.ici? b^ay r^pres^fitin^ 3 besad range of. 




17 



37 



V 
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SUCCESTIONS FOR WDIFYINC THE RECOGNiTION PROCESS 



•How CLOSELY DO THE FOLLOWlHC STATEMENTS 
REFLECT YOUR ACE?iCY*S COHCEPLNS FOR 
IMPROVING THE RICOCHITIQN PROCESS? 

Plijga circle one nusber after 

each SEattment, ■ ' ' ' 



DEAE 
41, 

43. 
44, 



ADOITiONS to PRZiEKT CRITERIA - = - 

should add ntw criteria co require agenclei to: 
^Verify data presented in an insc icut ion ' s ielf-jEudy. 



4S. 
46. 

41. 



Cqoperatr^th other aGcredltlng agencies in scheduling 
joint vlslti/when an Insiicutlon so requests. 

Provide edycational support services to upgrade pracclces 
in ihi field. i ^ 

o ' , 

Establish specla; procedurei f or evaluaang off-caapus 
pragrafss. 

Publiih policiei governing institutions* conEractual 
relationships wlch nonaccredlted organigacions, 

Monitor insEicu£lons or prograas continuoysly rather 
tfkin only after a period of years. 

Certify all site-visit reports and file, them wieh Dj^E. 

Gather sore spec If ic information concernihg the baek|round 
and integrity of the heads of Inscitut ions or programs. 




49* Conduct periodic surprise audits. 



13 



16 
'24 
30 

10 

12 

13 
16 



i 2 3 4 5 *6 7 



21 7 22 10 16 2 10. 

16 6 13 13 27 ^. 2 15 

24 4 15 10 19 7 6 

11 4 . 17 5 if 7 13 

13 4 12 8 IS 7 7 

11. 8 30 4 is 6 6 

49 8 18 1 8 2 

42 11 . 24 ' . 2 1 1 5 

46 8 19 4 4 2 i 



DELETIONS FROM FRESm CRITERIA 

DlAi ihould delete present er^certa that require agencies to: 

56.* Serve some tderttlfied need, as Indicated by the absence 
of a cotipeting agency. ^ 

'51. Hav^ public representatives ; on decision-making bodies, 

52. Have a visiting teas member present ac £he evaluation 
3' of Ehe report. 

53. Honltor laititut tens' afifirmatlve action policlci* 

54. Fosciir •thlcal practices by insticucloni* 
55., . Encourage innovation, 



28 22 6 20 2 10^ 2 10 

12 22 8 22 4- 18 2 12. 

11 ' 17 6 18. 9 13 9 17. 

21 11 4 10 7 11 13 24 

11 21 10 19 7 10 7 16 

8 22 12 25 i a -6 12. 



OTHW SUCCESTIONS ' 

If you would like to suggest additional modifications in 
the procedurei or criteria Isf the recognition proceas, 
please %^ite them on the back of this page% 



APPENDIX F 
XcciSilBILm TO OBSIRVATIOM 



HOW DEPtNDENT IS EACH STaVd^HT 
ON THE AlTUTENESS OF THE 
OlSlRVEH? 

FleaMtf circit one nuffibtr after yacR 



% SCUPC AND ORCAHlZATlONi THE CUKITY OF THE ACENCY'S 

, PURPOSES, THE EFFEUTI VENEb^ OF ITS 0Ra>^I2ATIQNAL ^ Median 

STRUCTUkE; AND THE ADEQUACY OF ITS RESOURCEB. ' Rating. 

20 The lyp^B ^nd levtlg of LniCicutions or programs 

accredited are cUarly daflned In pybLlshed doeumenta, 3,94 

21 Ttiu ditfficy'i purposes iid/or statutory authority are 

clejrly statL'd and maUw pyblicly available. ^ 3 91 

5 Kit' og^ncy 1*1 the only accrydlttn| or approval body 

fuf Its purpose In ICS region* 3^77 

70 Tl.y agency dafinei Its owti fiscal ntedi and eontrols 

Its Di/n budget. ^ 3.34 

72 ^ Tli* agyncy has chi financial resources to carry oyc 

Iti activities effectively. " 2,12 

100 Tilt* agency- i fees do not escted the reaionable coit of 

sy^taiftin^ and improving thu accraditatiyn procu^^. 3.25 

SO Ihy Hiaff 1^ able to carry out thu agency "d activities 

wUli cary and witlidyc undyy dylay. 3.12 

2 Tilt scQpi ,of the agency's accivlrlQs l^ neither too 
bruad lior tog llmlEnd in^rulatlon to the newd for 

appropriaie accrtditin^ or apprdvul itryicoi*, 2,57 

Ij2 Th# agency *s fee^ fof accred iiit Ion iiay prevent iome 
^chouib froa applying, e^peciVlly newly dyvelspiny 
institutions or programs. 2.57 

The agency ejctsts solely for purposei of dcrermlning 

eligibility for federal funding; it B«irve§ no ether 

need^ 2,35 

The a^ehcy*i accraditation or approval aiBuriS the 

expenditure of Mtydent and government ^neyi In. an 

Vducar lonally efftctive and economically pcoduetiva 

i&annerM 1,92 



fFigures in these four columns are percentagei based, on fj responses p 



AC»^iSiIi£LlTY TO OlSERVAtlON 



HOW OEPENDfNT IS EACH STATE^ffiHT 
OH THE AjrinENISi OF THE . 
'OlSlRVER? 



ACCREOITINC nROCEDURESi THE SPECIFICITY OF THE 

AlLLNuI'S PKOCEDUHES and rHllR PEATiHENCE TO ITS Hedtan 
PUaPuiES. . Rating 

41 Tli^ dwflnitlQni of taeh Itv^l hi accredieacion 

Of appfovai granted art cltarand well Uif ftrtntlaced* 3,78 

4a PfCkCedufii fgr grant Ing, dt^nylng, wlehdrawingi er 

ryln^Ejcing accr^dUdd ycacyM ac all lyv^ls art clearly 

d^^cr-ib^d, ' 3.78 

■ 3 ^ Wfien c ircumscaneei warrant Ic , Inatitutloni or proira&a 
ar^r f^^valyactd at Inc^rval^ shorter chart cht nari^l. 

ey^U. " ^ 3.52 

Ai^cfe J Itat ion or approval by che agency afliurfei thaE 

graduates are ocQupac ionally or prof ^Malonally qualified 

for appropriate emplQ/mtint^ 2,24 

571 riie agwncy'i riylaw of In^c i tut tons' or proyrami* arinual 
rvjiurcis are not ctioroyytt yiiouyh to ynbyre thaE ychuQl^ or 
Itrugr.ioiH adhyfti to ytJMdard*i* , 2.12 

552 fhc' ^i^unay'a proQudurei allow programs or Inscitytlona 

frurdum to u^e a variety of eduQdttbnal metliod^* . 3,27 

570 Innc i c ut iQiis of proyram^ afe fequlred to submit regular 

fEpgfCi indigati a cytitlnulng progroffl of ^alf-evalyat ion * 3.79 

51 I'tiu agency's accfedHting or approval pTOctdufti aft tlsQly 

and ytfg^tivu. 2*96 

-l^n Procvdyr^i for/ tl^^^aova fro® preaaeredlEatlon to fyll 

acu fwd 1 1 a t Ign are providud and appropriatoly r^lated> 3.52 

211 The 4gyn£y provides both forsal and informal consultations 
to preaccredited inatltyclons to aksls^ the= In attaining 
Jyll accredicacion. . 3*49 



HOki DEPEHOEHT IS EACH STAT£LPCNr 
ON THE AilUTDiiSS OF THl 
OlSERVLKf 

Pliaii circle one Buab«r mftmr m&cU 
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ACCESSlUiLm TO OBSERVATiaN 



261 



260 



.272 



603 



120 



59i 



EVALUATIVE PROCEOURlS: THE COHFkEllENSlVENElS AJ^D 

came with micw institutions or frouilams are * 
ivaluated, 

A Hrlf^sEudy and on^sica rovltw ^re r^qyir^d chat 

|>roviUe for Cha qyaiicaclvc Aise^gmenc of an 

ini^E li uc ion * s Of progra&*s icr^ngchi and ytaknisstSi 

Each irticitycton oc pjjjgfaE is given a copy- of the 
visit ing ceas*ii reporE , wfiich Indieaici i*treiiiEh3| 
M€akn^^!iiis p and recovn^ndai loni fur iaproVtsliATi-— 

VlilLing temB art conctrnad Wlih InitrycLioni 
r^iourcgSi aanag^sanE^ and scud^nc iQfvifies; ail^ 
foul ari^ai ^rt covered, ^ 

Visiting cea^^s engage in discuss long with facylty, 
adalni^f r^cofs p and scyden^^^ 

Inseiiuclons or prbgraf&s to be uvjlyyrod rn^y 
recom^nd ol r#jucE parcieular visiting t^as^eebers. 

The ag^'ncy E^mp^ra ies; tvsluat ivt role so as not to 
^lidangwr the repytatlon or the accrydltai lyn or 
approval scatys of Itg ih^t itu t lon^ -or programii 

The aguney usigs conpitant, qynlifled pgople ai 
pJrcicipanca on visit ing t^am^. 

Training ofvisiilng reas^ and "ffionltoring th^ir 
pjg^fortfiance asayr^i coniistune uppllcotion of 
atandarda^ 



151 The reliability of the visit Ing team report is 
quest ion^ble , 



Median 
Racing 



3.92 



3.66 



3,82 



3*81 



2,41 



3.19 



2.74 



2.00 



ERIC 



319 



ACCESSIBILITY TO OBSERVATION 



HOW DEPENDENT IS EACH STATEHEHT 
ON THE ASTUTENESH OF THE 

Pi^jdt circle Qiir nuffiber cer #ach 



STANU/yiDSi THE CUXRITY A^D APPKUPRl ATENESS OF 

THE ACCNCY'i STANDARD^ j^Q THE CAKE WITH WllICH ^ Median 

IT DEVCLOPS ^0 MAINTAIHS FHEH, ' Racing 



35C TIjtf jg^ney evjluatea Che validity and rcilabilicy of 



2.86 



334 Th# jg^ncy's tvaluatlvy program ot Its btandards Is 

wt:*ikj pgofly ijonceivedj or mafglnally *^tfgc£ivei 2*11 

572 T\ie agi>ncy oioniCQri continyliig cosipHance wich scandardi 
eliroygh regular riiporca sybmittijd by insticuEiona Or 
pry^rass* 3.63 

-^hZ JXh^_^_^^ncy regularly ravUws its ^cundaras^. pollyU^s, 

liic wfys^tcJ ^fQUpi^ 3t6l 

350 t^CjnJjrUs arc not revlsUJ until aftur inturii^ted gruup^ ' 

luvi? had 4ii gppurtyiilty. tiJ rcjct ro Lhii prupusted mvlSlons. 3.72 

3v2 Ail. a*^! ivy prot^rum'te impfovij accrual cac lun i'tjndards or 
evaluative prucedur*^ C^'g^i icadiy^, uuiifilryiiyusi 

wurk^hupa) carriwd aut, 3.40 

^11 Tnw jguiVGy'a scandarat aro yUur and dgtatlud anough co 
a^^ufy fakf and rellablu trwatmyni to instlcyclon^ or 
PTT^gr^iiusi ' ' 3,08 

156- The *i^yncy nonttQfa educaEiunyl ^candardy to keep E hum 

Qan-tf^Cynt wich occupat iuiial I f ynd^ . 2iSS 

332 tn^tttutiana ar progfafiis aro yvaluattd wich re^pecc to 

thwir own purpoa^it rachvr ihan in yompari^on with others . 2.97 

5S3 , ii jiidarda ar*? validated against thy perfori^ncg of 
gfaJujigi'i3( ayeredltyd irtst Itut ipiis, aucll as in 

liciin^ure txaainaclond. 2>i2. 

642 inforaution regarding atandjrds and guidgllhus for eonducc 

uf ^ity ^visits im esiGhaniiyd with Other accrediting agyncl^s* 3*57 

335 ^omy of Ehe agency's siandards are qutscipnabla or not " ' 
dymun^craccd ta be vaild or ryliabU, 1*96 



ACClSSIfllLm to OnSERVATlOH 



MUW UEPENUEfiT IS EACH STATEMENT 
ON THE i^TUTENeSi Of THE 
OiiERVEKf 

Pleiet circle' one nuaber mitmf each 



2S3 



700 



281 



1$2 



270 



390 



561 



,18 

580^ 
110 



RESPbHSlVENEiif THE SINSITIVITV OP THE ACE-NCY ^ 
TO THE NIEDS AND DEFINES OF THE CROUPS it AFFECTS, 
AWD ITS READlHESi TO RESPOND TO THEM. 

The agancy has pubiic reprtsencaclvii on its policy^ 
making bo<fies« 

All signlficanE, Ihcert^t^d groups are reprtsonced 
on the agency's policy-making boUyi 

Solicicaiion of the^;vf^^,^of scudoncs is part of 
ih^4|c§f tdi (aeion or approval proems* 

Hfesbcrs of. flpproprlace acadtstCj occupational, or 
proteisional fields havo a role in the accrtdiiscion 
or approval proc^ss^ 

Faculty .sanit'r 3 art givtn opportyniciti to make their 
vlewi known to rho agyncy, 

Written precadures provldt for fair and etmoly .trtacffiint 
b'y thy agency of eoffiplainti against insciEutions or 

^^rograffli* ' , 

The agtjncy is senile Ivt to technological or other foray 
of cdycational advancesi^nc; new knowledge and skills are 
translated into edycatic^nai criteria. 

Ho^t prograiaa or institutions in its field ot Jurisdiction 
set.'k accrtdiiatlon or approvMl froa the agency. 

Accreditation or approval by th^ agency Is the only 
practicable and widespread national 'cost of educational 
quality in its field/ 
* 

Experimentation and innovation are encouraged* 



400 



The agency selaccs its personnel in accordance with 
nondiscriBlnatory pract icea i 

The agency's public representatives hove vested interests 
in the accrediting. Of approval body or its consi ituontSf 

The agency lacks procedures to adequately review conplalnti 
regarding institutional or progransiatic Aalpracticea 
(@«8/# grade inflation, failure to grant tenure). 




Median 
Racing 

3.89 

3.21 

3.71 

3,64 
3.65 

3.69 

2,83 
3.18 

3,00 
2.96 

3.S4 

2,30 

2*73 



13 
4 



28. 



34 



19 



35 



24 
28 



28 



27 



40 
28 

33 
38 

36 

J7 
34 

27 
41 

35 

33 , 

36 



82 
39 
§4 

58 
59 

62 

25 
55 

37 

28J 

52 

12 

23 
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ACCESililUTt TO OaSERVATIOH 



HOW DCPiNDENT 11 EACH STATEKENT 
ON THI AStUTENESi Of TU£ 

Please eit^lm qm nusber after tAoh 



OUi FROCliii THE CARE WITH WHICM. PROCEfiURES ARE 
ITRUCTyRED AND CODIFIED TO AVOID UNFAIR OR 
' ^ ' CAPRlClbuS ACTIONS. \ 

kUl The chlsf tx^cucive sfflciir of chs evaluaCid 

initlEUEion or prsgraa li given an opporcunlt^ . 
tQ respond to ch€ vlslt"lrig team rapQrc« 

450 Ac li^J^c one camber of che vlitclng Ceam Is.requtrad 
to bm pfQ^Bnt ac ihs evalu4cion of ihm viMiclng 
tQAm rspQrti 

460 Accrudlc^E ion is vithdvdvn only afcar dua procisip 
for cauiep and wich •dua nutluii* 

sop Adv^f^d daeisiona ar# foilawcd by a hyarCng ac which 
rtfpr^^^nCac iy#i of fclty af ryrcyd in^c icuc iQn or 
program say appear. ' . " 

434 V Appcji procfdurea Iri^ludy yaroguard^ against blai; 
y>g.i no one tnvolvu4 in thw otlglnat decision say 
. a^rve on cha appeal comsiL^cOi 

590 ^ The agency 'i publi^had preeedures are^rfafrly appiiadi 
iti actions are conyiscen^ with iCi ptocadureii 

305 In ryviywing complainEM^ rha agency cunsidcri only 

iyb&tanLlaily suppar£ad yliegation^ chat cuuld impair 
In^c i(yc ional or progras qua! icy ot..af face ivanaii. 

493 AlLhough no ^pacific policy, or procE^dura ly scaCidt ic 
^i^ impliuic in clia ovyrall appoal prbdydufas §hac chs 
scacys of an insclcy^ion or progr^M can^oc ba changed 
yiiil^ an appaal id pending, 

71.^ Titrough iC9 structure and procedures cha agency 
eiisiinacai policically^baii^d daciiions x^t nay 
advariaiy affecc cha quaiicy of aducacion* 




Mad Ian 
Racing 



3.94 



3.88 



3,76 



3.8i 



3,70 



3.04 



2*82 



3.38 



2.43 



1 . 



16 



23 



29 



15 



36 
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AdCESSiniLm TO OBSERVATION 



HOW DEPENDENT IS lACH STATCISNT 
ON THE AJTUTENEiS OF THE 
OSSIRVER/ 

Pitaic clrelt ent nunber After each 



AGENCY INTEGRITY I THE. DECREE OF TRUST ENCENDE^D 
SV THE AGENCY THROUCH IIS RELIABILITV, AtlTQNOMY 
AND CONCERN FOrf EJ^ICAL P^CTICE; 

^20, The iigency Is n43t\cohsEraifted in its actions or 
dEcUlons by e les to a parKnc or ufflbreila 



661 



Thw ageney'a ay&horlEy is broadly recognlied by 
edycaiional tnsE Ieui lonu , appropriate licensing 
agiincieSi and proft^^ignal or occuparional fields. 

thi agency duniis candidacy and accrcdlcaEloh co 
insilEutlons or preifass Ic finda to b§ unethical 
in conduce. 

The agency deflneyj fosters, and Bdnltore ethical 
prjttXces by Inatitutions or progrioii with regard 
to such activities as recruiting and advertlMlng. 

, 72S The accrgditaEion or approvMl conferred by an 

agency has been dlstortud by pbliElcal expediency. 

710 The ayency perfoFms no function that tnterf-res 
i^ith ih^ ejcerciss of indypgndent Jude«ent about 
the educational quality of an institution* 

64 A ^Fitten policy or iot of operating proctdufgy 
cUafly guards against conflicts of intereit. 

113 Accreditation or approval rupreients Judgffients of 
qualified personnal; accreditation is technically 
; reliable. 

S5J Written procedurei clearly assure t^^t only quallfiid 
insiifutlons or pregraoii ar^ accrodited or approved. 

391 The agency** decisloni reflect fair and consigtenE 
^ application of its standards. 

7A3 Heabars^f the decAsion-aaking, body do not participate- 
in dU**u2aions or decisioni affecrlni Inititytiona 
with which ih^i aril af filia ted. 

326 Hynilorlng of institution orprograa practlbos in 

advertising, reyrmting, atudynt aid, or other areas 
is dyialled tnoygh to Blnisilfi abuSOi. 

.^38 Tht agency fills to ■enitor the frsuduUnt use of 
fcCud^nt Sid funds by Instltutioni or prograas, 

ail Concerns f^r institutional br program perfor^nce 

go beyond quest iont of •dycational quailty InEk^iiiygi 
iueh as athlcal praccica and financial ■tabillt^l-^^ 

• . , ' . ' - . . ^ li, • 

AcereditaEioh sr approval by^ the sgency is not s fsliabU 
indicatdr sf flithsr inscityt'lonal inttgrity of 9i4biliEy» 



.44 



The agtsncy cnsurcf that inati tut ions BslnCain toynd / 
cdycatlonal control of ^of f ^ciMpus or concraceed ^ 

""^gr4 




Median 
Rating 



3,12 



3.52 



3.16 



1.01 



i;a4 



2.85 



3.39 



3.04 



3,19 



2.87 



3.70 



2.80 



2,13 



1*97 



-2r70— 



10 



35 



^0 43 



^2 19 



15 



43 



.37 



IS 



27 



28 46 



28 51 



22 36 38 



49 



17 



29 



35 



22 ^l 



2S 



18 



26 



31 



46 



42 30 



37 



32 31 26 



63 



32 42 21 



26 39 20. 14 



31 .38 2i 



37 42 17 



FORM APPROVE) 

- ' . APPENDIX G O.M.B. NO., 51-579003 

, ' , ' EVALUATION OF OE CRITERIA FOR RECOGNITION ' 
OF ACCREDITING AND STATE APPROVAL AGENCIES 

Excernal Validation of Office of Educaclon Judgments 

For each hypocheclcal composite agency described on the, following pages: 

(A) Indicate whether you' think the- agency should be recognized, and for 
what period, or should be denied recognition. ' 

-\ Four-year recognition should be awarded, when /there are no defi- ' 
clencies, or only very minor ones, that woul^ Interfere with the 
agency's' functioning. jl 

- ■ ' . ' f ■ ■ ■ 

Three-year recognition should be awarded when the agency shows 
B^nqr d^lclencles that do not seriously interfere with its func- 
tlonlng^nd seem quite likely to be corrected. ^ 

Two-year recognition indicates that deficiencies are not quite so 
minor and. there Is some doubt as to tKel^' potential for edrrection. 

One-year recotnltion should be awa'rded ^/hen„the deficiencies are ' 
more serious or more nimerous, or, when you Have greater doubt as fl 
CO their potential for correction. 

Recognl^,^on should be denied when the/deflclencles lead you to seri- 
ously doubt the agency's ability to function as a reliable arbiter 
of educational quality. 

^ , ■ ff ' //■ ■ 

Even though the Information aboub each, agency is quite limited please 
•make the best judgment you can,\ circling only one of the alternatives. 

(B) Indicate which pieces of inforoa'tion about the agency were most crltl-< 
cal in influencing your judgment? 

(C) Jot down the kinds of additional Information you would have liked in 
making your decision. t ' 

.' - , ' ' ■ ■ ^ f . ■' 

Complete all three steps for one- agency before going on to the next. 

On the last page you will be asked to rank the 12 agencies, followina 
directions on that paae. ' 

- : • . . . ' . . ■ s ' ■ . - 

> * EducaCional Teatlng Strvice _ 

Barkeley^ talifortia " 
^ Jfnuary, 1979 

■ 3S4 



study la authoriied by law (20l^C 1226). Wiile you are not required to resptfpd. 
|»o^r eooperatipn is needed to mke the resiAts of this study comprehensive, accurate. 

Ml ioforma^ identification of the individual will 

^*B|^*|*l?elya>rotee by persons engaged in and for the puifposes 

FR ]r ^^»s*Mvm Uii viJ^ a^f^e dlgclo^ed Of ralia^ any purposls.' 



G»2 



AFTSR Rr:Ani;»u THE OESCRlP^rON OF 'THE 
llYPOrHSTECAL AGENCY lELOyi INOlCArE 
YOUR JUDuMENT OF THE NUMBpR OF YEARS 
THE AGEMCY iilOULD SE KtCO^UlZED. 

Cltcie one ol she 5 aUsrnacivtg to the rlshti 



4. YRS. 



3 VRS. 



1 



DENY 



EACH NUMBER IKS 
COLUMN TO THE CEFT 
,U.F THE DESCRIPTION 
C^KESPOKDS .TO ONE 
CHARACTERtStiC OF 
:HE HYPOTHETICAL 
ACENCY. ^ 

' CLrcjl^ no ::iora chan 
m^n£ in A above* 



The AdvLdory Board on QpchalmgLogy 

01 Is piCi-cloalnf the Office o f Edueat ion f or rtnawal of 
r a^e gnl t iori\ » . ^ 

02 ForoverlOyeariich as approved 

"03 baccalaureate and graduate programi In opthalmoloiy 
04 chrouihout the Uaited States. 

03 The Igard Lb asgldcad flnanelaLly by the. Amarican Advlsory^ 
Counoll on Eye Carp, Its parent organisation. 

06 Iti pol ley /decis lon-tnaklni body /Is made up of competent and 
qualified niembers, aad ' ■ 

07 ita adviiory council IncLude^ pub Tic represantat tvaa 

08 The d ee Ij lon-tnaking body, d o ea .no t a pp ea r to^meet frequently 
enough^ to function effectively. 

09 The dec iilon-ma^ini process usaa to -determine a program's 
scatus is publishad, bus 

10 the agency granES a probation status that is not defined in. 
lea published dscumencs- '. 

11 The Board undercakei Studies and sponsors conferencei to 
iinprova its approval standardi* : 

12 which are clear and detailed enough to assure that programs-- 
aretreatedfairiy. 

13 In addition, the Board provides written procedures that 
assure its fair and timely treatment of compiain.ta againsc 
programs by students and others^ It monitors programs' 

14 athtcal practices with regard to' advertistng and ractuiting. ^ 
prac t Ices and - ' ? 

15 also with regard to equitable refund' policies, ^ ^ 

16 When e 1 rc umi t anc as warrant it, prngrams are reevaluated more 
frequinilythan usual. 



Any raae^blance in the 
not Inti 



3f the hypothetical agenclea Co any actual orgartisatton ia" coincidental and 
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The Board'g ^ite-vigic p ro e u r c 3 ha v d bygn improved uv^r 
t h e y e a r S , 

Current proceduras require EhaC visiting too^s inelude at 
least one person who Is not a nonber of the a d p i n i i t t i v # 
staff or pol icy/dcclsion-Baking b^ody. 

19 Ac least one member of the visiting ceani ii required to be 
present at the Board's evaluation of their reporE. 

20 lefdFe^ this evaluatlgn, prQ|rams are given an oppoftUftity 
to Identify errorsMn the reporc €ind to lubmit supplemenca 
niaterials. 



UHAT ADDITIONAL FACTUAL rNFORMATlON AEOL^T THIS 
ACENCY. WOL'LD VOL liAVE FOU^^D USEFUL IV M\KI^'G 
YQL'R? DEC IS ION? - ^ - 

Please InJiciite below. . ' * 



G-4 



\Hv;'arHET:cAL auSncy below . lnd rcArE 

.^-'^1'^ JVr U!ENT 'JF THE NLMiER OF VEAAS 




RECOGNlTrOH (0) 



EACH NVmEER THE 
CULwMNUJ THE LSFT 

cos = £JpA^:os to or:s 

CHAXACT&VlSTlC OF 
THE HTFlTiHETrCAL 



-"^ni In A 



five. 



Tha An^rlijn Council on tacernAl MadtGlne 

01 4ppr^3ve^ proirina ric all Level g in this branch of siedleine. 

ic Is applying far rcnsw.il ot its recogn i t to n .3 J t er 
gj hdvlni approved programs for a.bQut 10 y^.-ifs, 

j4 Trie paclc ton JubsiLL^^J by Si ha Council iie^c:ribod the oc^anlia- 

:tsnal Jtru^tuta" md .j£aff a'lze, and 
:5 inJic^EuJ ch.ic cha \4^ncy ha^ ehe finanaial rysgurces to carry " 

t 1 c ^ 3 e t i V i c i a , ?i f f g e c t V e l y * 
^6 Ta^ Council ieiines Itg gwn fiscal n^eda and CD#itrglsU^l ova 

bi. igec . . ' - 

An' yr.br^lla afsini^aCLan a^olntr all.^q-berB of che pal ley/ 
J ac ii lon*riak ing body; - ^ 

a bfoud jpewir^^ .if aJua aCg^s and physicians is represent ad* 
The pcllt^yAitcl3ldn-^,akLng body la autonsmoui aAd Icj deetsiisns^ 
ara f In^l . ^ 
Thos^ ri^Tb^fs of the pol Icy/doclslsn-makirii body who hava 
p-irclc;=paEtd in the ayaluatian of a partlculaT program do ros 
vote on iin .approval decision for that ptQgtAm^ 

i 

Appall 5^ pcoc^idurei are defined and Isplentnc^d by the ^^mbfella 
o r g a n 1 2 i t L □ n . ^ ^ 

Advarsi? ap-rqval docislons =ay be followed by an appaalf 
*nirlng at wh tih r tp r a i^ncaE I ves of-tht affeccod p^psgram 
-^y appeal . ^ \ .. 

During an->Ute vtsita 'flatting teaaa angage In i iacuii Lons/^ 
vlth faculty, administrators, and stud ints* 
Thay enS'ire cha'c ihe ;irt^iovit process Includas the vlaws 
pi icudfnts. 

Culdanea, etthn^ chrough wrlcttn Instructtons or eons ul id i Ion, 
is. provided fiir the ielf-study arid on-'site visit* 
Th€ Caune.Ll's ^rttarla and p roc edur em f or swarding p raaccr'a dl'^ 
tat Ion st.iEui art a pp rep rid til y related La thass appllad 
for awarding full approval status* ^ * ■ 
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ri^ anJ the j^ency ..^nrtors, iHt^^i^h r^Kul^fly sub-itt^j 

Eh,? program's ceniinulng car^rUanae wUh tha CouneilU 

19 Th*^. agency pybl i^hc!!' the siandarJu by w h 1 pf^Q^f^f^^ 
\ ev^ luaE . and . . 

^0 Ln aJaiEU;n, ^alnEjins a pr^gtsA^ to ^valuAS^ rhe V4 1 id 1 1 y 

reUabiUtv of these seandar.di;^ . ^'^^ 

SI Th« C^uncU defines and ronlrorg e S h i c a I p r ae r 1 c e s b v progra^g 
vith.rea^rJ to rofunding .-utuon and' fees, pU<lng gfiadyo^^s, 

Us e^n personn^r ^r^ seUcSoiJ In.:iccgrl4ncc wUh nondiscrim- 
inatory prac^ic^^, ^ ' . 



AGE^ICi y-ULD vro- vuvc fir-H^ U^Ef' L i*: M^KI'^C^ 
Rlsase inilcate below, ' . * 



rME.S'VMBiR or, years . 



4 Y^f, 



DENY / 
RiCOCjaTIOH CO) 



-I. 



0# 



01 has a0€r#4 LCiid pf 0gr4^s. f or nearly t^tncy year*. . - . ■ 
02, Only fra^yaeei frea pT©iraiis is ^esredlcfl are 'eligible for " . / 

;j C^^nall p«tiEteniiig tha Offtsi sf £dUG*isian for rena^al 

■ - . f. ■' . ' ' ^ ' ' ^ • • , • . ^ .. 

for '4S£^aji^atlan li pfO€tiiii^d In.eaeh £*f ^He i years, 

'and |iiiva a 4#^ali^d o^^^ount ef Ics sdur^i^s ef rsv^anutr 
The .i^e.ney iel^sts its psrs^nniil In, aieerdanee i#tEh/ 
.ii^nd I ^gr inaiary praet Icsi'i but.^ - » ^ 

A n io tm^ t i d n i V P ? ^ ^/^^^ ^ ^ * i^^'^ i i £ is £ o f p e r se nn e 1 c i^n - 

/..yn4 4 r staff td '> ■ /. ^ ' ^ ' . 

MSfoey«r> s^Se iig^ncy's r« L'*£ iansh Lp wLth the As^riaan ■ % 

' rai4€ii ^yi|^^ioh$ ab^at the autenqsy if tha CQunsil, ' * 

^' ■ ■ ' • ' - ' ■ ' ■ f ' ■ . > ' ^ ' 

'39 Th^ aiffiiey apprd*/4ii snly pr^Sfcaig that ate Jeeatid In Regionally 
«€§reiri'ted InstitijitLini , r' _ 

4f)d monitors .the a@siplian.€e €if these programs ^ith its atandmrds 
. thTtily'^h reportfs whleh approved peogfasi suat subalt rtgulayly*^ 
II ilHen a pr^ir^Q fatlJ t« seet the Council'i ftandardst the afaney 

haj tr^ndafdlied piroe^^dufei fo? Withdraval af approval, 
\2 Cthjlaat. pra£tic«f^ h&waVtf» arc soiiltored at th4 l^s t itutldnal 
level by:_r>#^^imT 4e€redltLng ag#n€lei% rather !t4iaa by tha 
^WtTHcII itself. ' % ' ' * 

1-3 fhi kfeney provldea written tniitruat loni and sakei €onjuitatlva 

services avallii^le^tepfogfaai hef o rf ^lel f ^atud Leg aeii begua* 
1^ thaia reiiources are alpo available to prograst and vliitlns / . 

ti^aa mis?!ib«r4 in ^prtpafit4Jsft= lor^^^p^^l tit.j^ 
13 the age;n^y atruetar'ai, if lf*atyd lea by provid ing a queit isnnairc 

that prograa^ ^^ust eo^plete* \ 
l6 However, an eicamlnactan of the teli*»tydy queit tonnalre • 

Indltates that fttidentf are net Involved In tht«i phase of . 

thit^approval proeeif • ' ' 



The agency llsc^d detailed critarla for the ^^igcUon of the 



Programs may rtcommind Of relecc partlculir team sewbara prior 
. Eo^ttki on-il^t v^lsl t , • 

:^ DEAt staff obitrvtd oni on« s i e i Vit 1 t and no t t ha t th^ 

vlslEing team sought Input from all appropriate groups. 
= - _ . ^ v.. % ' ■ ^ 

fO The ajtney hai puhltshed rults. d f p^octdure regarding apt^ale, 
21,.gtailng that U ^ d^tiBi^n la.mada noTto approve a program. \ 
repreientatlvea oi/that progran say appear at an appeals - ' 

22 The agency publlahei the itandards by which it tvaluated ' 

prograBs, ^ . V ^ ^ ^ " ' | ^ ; 

2J"'^byi ir did not provide informatiQn about Iti mathods of 

^ssesslni the relUbfillty afld validity of those standards/ ' " 

^4 Although pfograms are given advanct notice of .propoiid changis 
In standards, there sttffls to be no set procedure for them to 
' c5r.:5enE on the changeg befara chev are i^nl ' 



G.;kKAT AQDItrONAL FACtDAL rNFOHMATIOjl AiOyT^HlS 
YOUR DECIflON? \ ^ 
Please indicate be l^^ws 
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A. AFfSR THE DESCREFTtON OF THE 

HYPOTHETrCAL AOXnCY BELOW. ^ ^ICATE - 
VOUR JUDGMENT OF THE NUHSER OF ;YEARS 
% THE AUENCY SHOULD BE ' EECOCN 1 2 ID . 

• ont df ehg 5 al^tarnaetves cg tha right* 



'4 VRS. 



3 YRS. 



Z US, 



1 YR; 



.'DENY ■ 
*lffiCQGNlTlOH XQ) { 



I* EACH NUHBgR irl THE 
COLUMH TO THE LEFT 
dr THE DElCRLPTtOK 

' cetrres.pondi to one 

CHARACTERtSTEC Of 
THE HYPOTHETICAL' . 
AGENCY. 

Circle no mora than 
-'^ Che nun:b#ra, 

tnriuenc^.: -our Judg^ 
m*ine In A above, . / 
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U ha^ baen aeertdlclng B.A. and iraduat^ level proirass 

02 for over 10 ^eiri* / ^ 

03 It L§ ptciMonliig the Offlct dC. Educacton for renewal of - 
. rtCQgn I c lQn% ■ ^ - 

S4 Aselse^ed by an umbrella orjanl^arion , fhe aiency ^hai thf . 

03 financial resources to carry sue ICi otfClvicle^ efcecrf^ely, 

7^6 Each of Che FaderattQn*! levels of progran appro vai is-. 

clear and well-d 1 f f erenEla ted , 

Q7 It does not, howeveTp publiih Informarion abouc one of Irs 



i 



approval gcaeasei=* 




OS Ourlni atte vl3lEs^ v,ls4t'lni Ceami calk wlrh f acul 

adj^i^iCracori p and icudenrr, ' . ;^ 

AfCer Eheir viiir rhey provide che program wich a yrltcen 
report dUcusstng irs SErengthi and weakneiiea and making = 
racomciendacloni f or Improveaen t s . 

10 It ihould be noted chic sire ^isic proceduf es have been 
revised within Che pase few years. 

11 Should a pregraa appeal an adverse approval decision, the 
Federacion cannoc change ics staCus while, che appeal Is ^ 
pending. - ■ 

! ' ^ ' . ^ < 

\1 The Federacion rev,ltwi:i&^s s t and ards policies a^d procedures^ 
by ItivtEing suggeittons^f rom va r lous' in Eer es Ce4 groups , 

13 Furthermore, scandards are nor reviied unCll th'Vse groups 
havf had a n oppb r cunl c y C o react co .proposed ^rgvlilons,. 

=14 The agency's acEloni are conaisEent with* Its published 
proctdurea * . . 

15 For Instancit programs are evaluated aC InEervals ihorEer chin 
the normal cycle when c IrcqmsEances" w^rranc Ic, 
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ta . Ethical practicas ty programs are fogtered and mofli^rad In 



idvtrai ^artas 

16 Ftcrulclng and advt 

17 placiment of gradu 

IS Henber^ of Che prof 
19 as doas the publie 



agency's advisory dounsils 



20 It should bt nottdj 



rtlslni practicei , and 
£ts ■ in appropriata gmplqysie^it . 



tssien have a rolg in' th^e approval proceiii 
Public ,repf eatntaeiv^s serve on all of th< 



Eh^E the agency*! S^licy/deciiion-making 



body eonvahes CQo^infriquently t o f line t ion ef f tc t iva ly 



WHAT ADDITIONAL FACTUAL niFpRMATIOM Aiptit THII 
AGE!;CV WOL'LD YOU HAVE FOCND USEFUL InAaKTSg" 
TOUR DEC IS 10**7 ^ 



Piaasa indicate b'alow* 
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^FTER READINO THE DESCRIFT tON OF THE . 
IIYPOTKETICAL AGIHCY BELQ^, I^iDlCATE: / 
VOl-R JUDCHEHT OF THEm^iUMBER OF VEARS 
THE ACISCV SHOULD S E^ EECOCN I 2ED ♦ 

CircU one of 5 alCtraacivfis to cht rlghti 



4 YHS. 



3 YRs^ ' 2 yas. 



1 YR, 



EACH ^UHSER IH THE 
COLIMN to TKE LEFT 
OF THE PESCRIFT.ION 
CORRESPONDS W 03 



02 



THE HLPOTHETrCAL 
* 

ot numbers, 
faatari that most' 
T,yrt£ in A above* 



05 



06 



07 



03 
09 



. 10 
11 



12 

13 
U 

Li 

y 

17 
18 



tha Assoclac^iQn;'rQr tha Advahcemi^t of Automobile Taehnology 

■ . ■ ' i' ■ 

. ■ ■ ' _ ^ - ^ - -t: 

ii pnElttomins Che Offici of Education for ftnewal of reco'gnttion. 
It has apprdvtd prQ|rami at- the A. A* and B*A*' lavel ' ' 

■ , s ^ , ^ ...... 

f.tfr about 10 y«3Fs» ^ : . 



^—tr ^^ edtly dggtrrg s t yp qo an d e-l-s -^ =pjLa. ^^ t ± g±h la fag 



accrtditation in iti Ac ct.id 1 ta t Ion Manuali and / , 
only iraduates f^om proicimi it appro vas are il^i^lble for , .■ 
employmant in the frid, , 

s r ■ '■ - 

Ths pat it ton submit ced d ascribed the agency's annual volume of 
^'otk , , . . ' " . - V \ ^ ' ' ' / 

also providgd LnforniaE Ion reiara^lng. scatf q ual i f Ic a t ions ^ -p 
which:^iuf giits chac ' - ; . - 

th>. Association may riji inadaquacely staffed. There is ' 
iomt doubt ai to whether the agency f unc tioni indipen^ ant ly of 
Eha,jkciaty o f Aucos^t ive Engineer s , /the parent organization ■ . 
wf|h which it is. af fiilatad , The agency does, however, i 
hiive public rtpras^i* attvti on ics pollcy/daclslon-raaking body, 
and it .Sfllacts ^ali meabera of thiBe bodie^j as well as \ : , 

visiting team mtmbars and cojftiu 1 t an t s g In accordancfe wish 
ngnd incriminatory pr act ices . ^ . 

Only programs* located In rtglonally accredited IniEltutlons 
art elt|ib'li for ipproVal* ^ ' ' •. 

Approved programs ar,e re.vtewad at raasonable intervari,' and ' '/ ■ 

Ehelr compliance with the 'agency ' i i Eandards Is eniurrd by the 
frequent raporEs.they submit to the ^Asseclat Ion* , - 

Althoygh programs are ftvin advanca notice of changes In % 
standards, they are flat given an opportunity to eommene ,en the , 
proposed changes^ = ^; . 

In addition, It Is not ciea,r that the agency has an afifecc^lve ... ■ -.^ 
rnethod f ^ r ■ a 1 s e i s I n g the reliablllEy or validity of lEs ; • 

"standards* ^ ^ * - . - . . _ ... ^ 

When a projraa falls to meet Its s C andar ds , Che ag'^ncy has 
procaduris f or . wlEhdrawal of approvalp 

Eehlcal. practices, however p a.re monitored at the Ini e 1 1 u t lonal 
level by reg lontl accredltlBg agenclas , rathir than' by 
cht Association, , ' ' 
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19 PfOirabi are requirfd £o cosplice one of two alcernatt 

quesEionnalres providid by the agtncy f ©,r conducting their' 

20. Nowhere is chfre an estpllclt rfqulrement, however, chat eieher 
faeulty or itudents be involved In the ielf=icudy* 

21 The agency dticrlbed pFocidurei\f or the ae! ec t ion o f vis 1 1 Ini 

c e a m m e n b e r § . 
,22 Ehat allow prograias to recomffiend ' or rejfct particular team 
mesibtri prior to the ftn-ilte visic, ■ ' 

23 The DEAE staff meriber who observed one on^alte vtilt npced tWc 
the viglcing ieam talked with all appropriate groups* 

■ " = \ " '' ''' ' 

24 Th e agency has deiailid procedures and criteria for wichdrawing 
approval, and doey so only af ter not if y ing the prograp* .^v 

'21 Moreover, the agency lives the program specifie reaions for %ny 
adverse- decliions it say make re'iardini the program's appeal. 



WHAT ADDITIONAL FACTUAL IKFORMATION ABOUT THIS^ 
^AGENCY WOULD VOU/HAVf FOUND VSEFUL JS MAKIVG 
.VOUR\ DECISION? . . ^ 

Please indicate below* 



A. AFTER READlNa THE DESCftlPTION OF THE = • 

KYPOTHETECAL AGENCY 4EL0y. IKOICATE " . = . : 

YUUR j/jDG^ENT OF THE NUMSIR OF YEARS . ' ' / . , 

THE AqENCV SHOULt) SE RECOGNIEED. ■ " 

CtrGlf ^ of the S aitirnacives to Eha righc: * 4 VRS, . 3 YRS, , 2 VRS. 1 YR. RECOGNITION (0)^'^ 

■ ' ; . ^ The Assoc isrion- of Campusei . Wl e houE WalLi ' . . 

-.^ . ^ i ' ' ■ ' > ^ ■ -. , ^ ' -. ' , . ' . r - 

1. EACH NUMIER^ IN THE Ql- accredtti pottseeandary ins E 1 eu £ loris chac offer art alCernailve ^ 

COLUMH. TO THE LEFT u ^ ^ i Vi - ' ^ 

^ orgcRCPTtOM ' ^ E f ad 1 E lonal-^ sollgge ■eurclculua, = . 

---CORRESPOMDi TO 02 I E haj^'bgan-aeataU^ InatltuEloni for^Eha 

ClfAaACTERtSTlC OF / ' ' " " ~ 

THS HVPOTHETICAL . P^^^ ^^^'^ y^*'^^* . 

.AGE!*'CY* 03 Is p6 E 1 E t o n 1 ni -E ha Offing si Edueaclon for 1 ft 1 1 lal . r ecogn 1 § ion . 

'CtrcLa no ^of€ Eiian ^ ■ 

. riv* of ^he nu^nbars, AgsociaEiofl is a no p r o f t E id uo a E io na 1 organiza Eion * wiEh' 

Eo iadUiata Ehoi^ OS an annual budget for aeereditini acilvlEiea of undtr 125,000, 

-\.\-'.^ors ^hac *^osE ' _ _ K ■ > 

. tn:.lyans6d' vpyr Judi- '^^ Tha^e Is evldinca Ehat ;iyg,j|«,3€s. chat agincy staffing may ha 

in S 3eove/ . ' inaai^q'4ati for its workload, - ^' ' ^ ^' * ;^ 

07 IE do BE riivaluata Inst it ue ions every 5 Eo 7 ye.ari ^ although * . 

^ , ^ ^ 13 =1e doiii noc appear eo inel'udt datii of na^E schaduLgd reyiiws 

• ^ in ity ^ubliast toni , . / ' " " ' - ' ' 

09" The Lvpas and levals of instltuEions Eha agency accredits are ^ 
' clear'/ dif inad -in its pubrished 'documenEs . 

' ■ 10 th^ A^'s.iciat ion describes acc r ed lEaE ion lEaEuits chat are not 
equivalent to ihe p'Veac c r.ed 1 1 a t ion s Ea Eus d e fined by Ehe 
; - . ^ffica of education, / . ^ ' ' " 

i ' ^11 It pub Ehe dacisi4>n>-maktng p^rectsi it usii to deteralna 

: ^ ' 7 Nhe status of InsttEUElonsv^-- . . _ ^ _ _ _ 

12 Pubitshed proctdures/lnelude the' safeguard tha t an, insE ttuEion^' i ^ 
' * statup nnot be ehangad b^ the agency while an appeal is - . x 

; ' = • pftndi^r.. .-^ ^ / ■ - — . . , . " ^. 

' 13 but is dn^iliar that an instiEufion may, appeal an advtrst J 

- ^ Ey.aapiet of iiis E 1 tuE ional, self-studies iuggast that the 

^ li agaftey*s ^^^Lf-stady procedurea are deficlenE in tha t Eft.ty . 

n^E « prov ide"^^ f or iufflciently crlEical a'ssessmtnE of ' 

ins^i^jtiona " atrengths and weaknesses* , 

• i IS - or for an asiasifnenE of the accosprishffient of Inst 1 t^tlonal 

^ goalu^ / . • \ . ' . ^ - ^ . 

, . , , " . , " ■ ' . ^ • -5 

li The AsBOci^fcJl^on has no wr i t ten pel ley ragarding the ' ieleeclQn; 

of yialt ing' €etm 'm^flibers*. ... ; " / , . . a3 

' 17 tt/hasi howev^lri provldird docuoenEs ind icacing the qualiftea^' 

" t4onf of Eha current tea® aieBbers* ^ 
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IS Vistting ceam4..^^clud^i at^ltajt one person' who ii not a roembir 
Sf th# agency ' s "ipol IcW'dtc iglon-making body ^or adminii trativa 
staf f '. 1 ^ ^ ' ' . - 

19 Several cons c Ltuenc lei art und emp'T t sen t ed on eht a|en«y'i 
pel iey/deeiiion^making body / ' ^ 

20 For Initance, tht agengy. does, not provide fer publlG repre-r 
stntaElon on that body* 

*1 Furthermore soma doubt exists as to whether th« agency ensures* 
impartial and abjpetlva avaluatloni and Jecisions , ^ 

22 Ko formal prodedurii exist for inselturlona to. comoatit on ^ 

reyiiloni of standards prior to thiir adoption. , 
-5 Evidjnca of th« agency ' s acceptanct by colleges in its domain 
' is quest lona^l e ; ^ ^ 4, . 

24 some have coaplained that the agency is not serving their needs 

25 Infarmatian about the agency'f effectiveness in a'ssesilng the 
rellabi'lity and validity of its standards Is inc onclusive and 

26 it appears that the Association does not mainEaln an activa 
program to evaluate the reliability and validity of .its i: 

* standards* , 

27 With respect ethical . practices i the agency's ^ tandardg.af e = 
not clear in the'' areas of nandiscrlmlna cory. admiss ions' pal Ic its 
and ..equitable tuitioii refunds. 



:^WHAT ADDITIONAL rACTUAL IHFORMATIOH ABOUT THIS 
AGENCY WQULD VQU HAVE FOUND USEFUlVin MAKING 
'VOUR DECISION? / 

' \ ■ . \ ' ' \ 

\FleaiJi indicate below» , 
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A. ARtER EEADE^O THE DESCRIPTION OF THE / \ - . 
- V HYPOTHETICAL^ AOEHCV piLOW, INDtCATE . , :. ' ' ^ 

VOUR JUOeMENT OF THE S'UHBIR OF VEARS * . - = 

THE AOENCY SHOULD SI ^ECOCNISiD. ■ " _ , 

, V ^ . DENT I • 

. Circle £nt «r'^-^ ^ al6gfnaciv€i Che fishE.i 4 YRS. 3 YRS, ■ 2 ^VRS, 1 YR, RECOGNlTIOH '(0): 

, ' * Coniat££ee on Rtsaarch in Mtd^^c^l Tachnology "* 

B, EACH- NUMBER tN THE 01 has approved programs ac all Itvals In £hii medieal field 
COLUMN TO tHE LEFT - ir. \ " 

OF THE DESCaiFTlQN , ^^^^ ^^^^ .20 yearns, iCi peEltlon . ^ 

CORHlSfONDS TO ONff 01 for renewal of r aeo |n t t ion ind lea Ee i that 

CHARACTERESTtC Of - . ^- . , ^ g j*. j , . " 

"ME HYPO "HET £C \L ^ ^ prtTax^^unc C ton Is ac c r ed 1 C a C Ion i and ' ^ 

AGENCY* -_ 04 that it da £ Inei Its own fiscal naadi arid controli 



f^<--ij. « iti own budget alchouih It. is 

Llfci^ no ciore -nan . ■■ ■ - , . 

llyj^ pf tha nuf^berSt Oi ass is t ed*^ f inancial fy ah umbrella organization. - 

Ir^i^'atS thosa ' ' - . ' ". ^ . . 

..u » ^ 06 ^hfi petition also lists Che qualifications and selection 

Influancad your* judg- prociduras prescflbad by the Commit'taa for its governing,^ 

tnen^ in A above, ' . . ^ 

board sj.embefs, * , ■ ' 

= " ^> - ^ '■■ . ' " ' ■ 

^ " " . 07 to be eligible 'f or approval j a pro gr as must be located? In' 

an Ihicltutlon that hai been a ccr edited by a re^ognlzad^ 

. ' agency other than the Comfatttaei ' . 

' * 08 The Cdmmlttaa doeinothave a prea cereal t a tlon atatusi 

" ' but it 4oas hava a provlslo.nal itatus, 

, Of 1 The agency requires a proiram leaking ^royli lonal § cat us 

^ \^ . to C0 3IP late a self -sturdy and to undergo evaluation jby 'a 

'\f y is i t Ing t aam , the Comalttae's petition states that. 

l^^ Its salf'^itudy and site visit procaduraSi as well asi 
\ ' 11 1 1 s s candard s hava bean r a v^ised or r a f inad g and 

^ \ provides tnf 6rES^t.l'en about . ^ 

\ 12 Its; itandards* development : process i Thus i -progr^as to be 

\^ .13 avaluat ad laay reeomnand or reject particular visiting. 

team roemberSp and ^ • / ■ — 

^s^ 1^ . when clreums ta'Tices warrant it, tha agency will reavaluate 
_ pregraras at .intarvals shortar.than the normal cyclei 
tS^ Solicitation of the vlaws of sTtudants ha^s become part of 
the" app^ovai process. ' % 

' 16 MeAbaifs of the umbrella or lanl la t Ion pro v Ida ceniulting ' 

,5 serv.lces to.prograss. 

17 The agency nalntains a list of thasa outside consultants» 
IS and it takes the public Interest into account by InclUiding 
public representatives Ion all agency committees* 



\ 



'3.? 7 



.■•I 
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19 "The Committee has published rules of proc'edurf regarding 

nppep^lg t and , it^ 

20 prohlblta cht changlng_aL^a\prograia' i geatus whtlt an 
appeal li pending, 

21 The agtncy has als© ; publ 1 shad a\itatemint affirmlni 
adharence to athleal^ ^ratclcts aM it also requirag 
prograas to- do ib, ^ 

22 itm standards are cl^ar and datailik enough to engii^a fair 
and reliable tftatment all progra^m^ 

23 Basidas requiring programs co-.^gather InVorBation on pfOirao 
e^raduatei, t'ht ComBlttae 



24 ^malntalns ^' ptogram to avaluate tha validity and reliability 

of itg ow,n aducfational standardii ^ ' ' 

25 ^ectari submlttid*i^lch Eht p<iEition indicate that the ag&ncy 
is recognized b^ '^thar recognized - ac'cradltlng ai^ncies and 
by tht..Councli. on Postsecondary Accreditation, 



WHAT ADDITIONAL FACTUAL tN^^RMATIOH ABOUT THIS 
ACENCY would you have FOUS'D useful IK HAKlNG 
YQUR DEC tilpN? . 

Please indicate below* s 



■\. 



AFTER RSAOiNa THE DEStfRlFTtOH OF THE 
HVPOTHETtl'AL^ J^CENCY BELOW/ INDICATI 
YOUR JUD«M£NT. Or^tHE number 01^ YEARS ' 
-THE AGENCV SHOULD BE RlCOCHlftD. 

Clx^cli ong_ ot' chm I alcaf natives Ep the fighE? 
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YRS. 



3 YES, 



2 YRS. 



the Coun&ll on CdrporaCfc Admin li£ fa £ Ion 



\ 



-DENY 
EECQGN'ITION (Q) 



£a.CH IN THE 

COLUMN TO THE LEFT 
OF THE D^ESCRIFttON 
CORREfgOS'Dl TO 
. CaARACTlRl^t tC OF 

hVfotketical 

AGENCY. 

C Ira 14 no mot^ shin . 
to ifiicasa %ht33k 

^i|?ne in A above* 



01 

02 
03 



04 



0^' 
07 

09 

to 

12/ 
13 

U 
15 
16 



aaarodici buslnais^orlinced Instisaclopa ac all pos t secondary ' 

levels* . ^ / ^ 

_lt^ hii b%en accf E ing "^rini £ i£u t io.ns foj less than ftvt '/^ars* 
and jL*s' petLttonlng Eht Offica of Ed uc a Clan for Initial 
r ecogni t ion *~ '"7^"^"^^ == ^ 

Thi ^eclclnn IndleaCed char ch^ Cciunall if adequarily gcaf^ed 
foe tEi .if f a^civt funat ionlng * ^ ^ ' 

"iCi fliaal neads are sftXf ^^def Intd and its budget is seLf'^ 
c o n o 1 1 a d * ' ' . * , 

A" parafte or lantiac Ion appolnCi all members of CCA's advisory 
board . > . ^ . , \ " ^ " 

All iiinificanc bonfclcueneies are repV|sennd on £ha agency's 
'pollcy/^i^aiiion-makifi'i ^pdy , 

Ineluding .a wide range of 'eduea.cori and bysine3>s peopTs^ ^ - 

the pol iey/d^^sion^^aking body li .auconosoys^ and ics deeisi'oni: 

are^ f inal * " - ^ " . - • 

The doun^il * s '^tr sonnel' art fttleeced in aecordanes wich 

npndl#§f latnacory praecieei. ; / 

' \ ' '. ' ' * ■ ' . " ■- . • - \ . 

[n addi c ion CO . iei . other Ca.iks ch€ ^parent 'organisa^ ion dtf inei . 

and^ iisplimenca Che appeal ' ^roeedure^g 

npciiying ini c itu £ iofii of che tfpecifie reasong for an advarsa 
a€Cion o^, an - appeal . ■ = - . » 

Heabera of>t)i« pol iey/^de€ is ion^aaklng "body who #Vaiuat« a' 
par'c|^ular tnicitution may noc voi9 pft<^ the^eeredieing d«cliion 
for iEha£\ins£i£y£ion. ^ ■ ' 



The Coyni||ii~a pybiiahtd praaedures are fairly appiiedi i^a 
ai^clona ^re ^onsiaCenc wich ics pro^adyrea* ' 
tea definCcldn of <^a«^hr level of aG^r.adicaEldii is gltar Atid 
aall-d Lf^ereneiA^edi ^ . 

Furcharmof Si let eriCerlA and , procedyrea f or? iiwa>^C ng~"~ 
pfea€£Te4ica%i^ir^€aiiug ar« apptopr i4£^ly rtlaiird to zh&m% 
iipplied for i|«rtfdiHi* ^edrtdlt#d fCACus, ■ >^ 
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17 



18 



19 



20 



Each InitiEutiofi appralias cht aceoffipllihment of Ies sEacid 

©fej dccivas and " ' . , 

Che ag#a§y monitors CQn^lnuini cempllanfe* wlEh its sEsndards 

through rciuiar reporet sulmlt^td by fihi insEltuEl©i\. 

The Csunsil maincainp a pr^gras to evaluaej tlie validity and 

reliability of its sEandardi, 

and rjivlse^ Ehea only afitr .^nBertSEtd graupi havt had an 
oppofEunlEy to rta€E Eo Ehe proposed revis ions . 



31 VLsiEing seam sesbsri are Erslnad in workshops and i£s|laf' 
agEiv iEiei I 

22 and are inserueEtd to inglude Eha vlows of sEudenEs i'n the 
ai^e^f tditaE Ion proQtss* ^ ^ " ^j^- :r---- 

li The agtney d€f Ints* foiEerSi and aoniEorg Ins f^ruE ions' , 

frEhlcal praeEiees wish regard td nondAiei^iffiinaElsii in adsissi 
4nd ^fiploysenr^ ' ^ . i 
24 refydirtg EulEien ^nd feei^ placing gfadu^^es, Hhd -proviii'ig 
= t r afisari pii ^ • . » " ' , 



WhAl^ ^DOITIOKAL FACTUAL tNFO&MAflON ABOUT THIS: ^ V ^ 

AaE:^CY WOULD HAVE FQUKD OliruL IJ5 MARlKO ^ ' 

j;fFltEflit.. Indies tt ^Ifelow, M ' © ^' * - ■ ' ^ 



AFTER si:\pn;d rHE DistRrpTioN OF riiE^ - : p : 

VOUP^ :iy:D^nOT^0F THE MUHSER OF YlARS ' - , . ^ . - ■ 

THE A^giiCV ^HWCt? Si ^ECOCHIEED. * ■ ^ ^ J? - ^ ' 

\- • ^.••^^ ■ \ . ' " ' - . . " • , ■ . • 

, . ■' " ■ ■ " • . : ^ ' ■ , ., ■ . " V 

' ■ ^ Th# N%iEiondr A3,^3c la t Ion of CQuns«L^rs for ehs Eldarly ^ 

• ■ ■ - . . ' ■ . ■ V . / 

■ j '' ' - ■ ^ ■ • = ^ -^^^ . = • v^' 

!:aCH SU'*3ER is* the > .ipprov^^s bace'slauf eaLa and grddyae^-dair^e level progra^^* for 

cOL^;?!^: tooths left ■ * c,' ^ ' ; = 

i?f 5 llCit ^ ?t 1.0,1 ■ 5 f 4 l^ag e f eounsa lors ©f oldor pac^p le ; ^regraas usually / 

fJ_V£-^?f the nu!^^^rst niti^n hi th4 ^iti^^ of SduQtitiom. ' - ■ ' . . ■ 

* !iv^^^ "r'" ^^^-*f i " 04 Tha A^enev'A njElonai fiCQp^ is ^l^arl y d^^tned . and Lts r 
f.^?aE in A iKn*^, r ^ ^^ipporr lr^g docu^^nis ^sai^^^hai ^ha » Asio^ l^Cion It ^ a / 

" provided Ehf ag^nc^ £ lea lEs annual' ^udg^t for ^^^^^^ =^ ^ 

^?;er#dl^ Ini jeetvlEliii as uadir 125,000* r ; i 

\ 0?- tn*ui' f U [^Fi£ e^rimlE^iint of f^rsonnel Ci^€ -OT ifi^d^quaEt /^ 

\ % . . ' _ 

^ ^ ; * ^ ' OS rhe i\jiseiiiEion*s ptoeedur@s eniute^hac £ha f I r si «va f uaftlon --^ 

' / ^ ' \ of 4 pcogria. lA/ inlf l^^ad: la re'lpond^ to an ap^ lies t i^a £ ros : 

' . Ehs ,^hl€f #>c#€ucive offLa«r^sn^.€h3S ^ ^ 

.1. Of ^rd^ra^^ ar« raiyl^rXy reviVwsd at * ro^sonAl^a tatei*if^lJ .-^^^^^^^^^^^^^^^^^^^ ^^^^ '^^^^ 

10 . H^^£#«v«t » chtt ^at#'s thft n6x£ ffhi£dui«d feviev o^ ^lasr^ditrd 

programs ar* noE ^ybll.th«d< lE^'hoald be noead thiE ' -^ 

<eqiiiv.il^nt £i2 ttsos^ uiod by iht Office o_£ gda€3€fof^. " 

. - ' I* Ix appiiarf EhiE i«l£^i.iydy prosadiirci ara defLcliinC La eHac * 

■ ^ .eh^y de^not provida' for Aa^lytle^asseii^ea^ of prajraaV' ^ V 

J IJ f©r an at ^^sii'^tns of ' a^eo'ipl L.f hsifiiiE proir^smAeic^^ ^ ^^^^^ i^^ 

: V With f;«»p<ie^ to vltiie procedures , agencvi pol leiti for ^^^^^^m 

'£h« 3«&«€lil0ii of visUini ce^a^ ^^Dt«f s dr« iin^l^art ' ' . '"'l 

15 th« AitfOG4d^ibnh<Js provided t/i^^^i lad Leafing ^hs^qu^^ 

^' • v' - f i^HT-^ af ;the ^urrf nS;^\vl 3 ic Ing 'teaa; is^fj^ers ; >^ ' ' ' ' 

16 Tti« «igti\ey*s poiicy/d^einiOR^^dkLiii body do«f nQt^^%lXf^s^t^\^'.. 



f©^-^^ ■ V ^ IS; rtscriettd c© ©ne area of th^ ffouncry, an 

^ ■ I . ■ pf©gf.i£5s have ce^pl^ined this. th¥ ags^ncy doei not 

- ^ ■ ' ^ f0r pregr^s input, -^-^ 

A ; V ^ »^ ©r If nbe ©th^rvist g^rvirtg i^hnir ne^iii^. 

v'/ , ' . It F©F insian£#, tjie-e are no forrsal pr©ct*J..ri?s r^'^r 
' ' p^©graas E© c©miJnt %n rei^li'tons ©f sta^dar i^ pn. 

■ ■ ^ • • " ' :t heir -adopc ten. : 



of 1^ 



The infertile ton the Aisseiatlofi pr©v!dc4 ^ 
of ^.f^€Sisln§ the E^'ti'ibillEy and v^IfdUy 

th^re^ls no ferial ptto^ra^ cd^ ^va I i t . ;!vyv: h rtlui 



- " Th^ agen^^ report* Jthe S^jielf r a^sona f ^ijv. r , 

- ^ ' ' «^^^l«l^ni e© tht chtaf pr©grass cjti^cy? i anJ . 

^i? c a ^ ^ppecEunlcy £© apptal , y-* 

I * / * «3*^r©ce^ure»*/©f appeal Ji^ ho*i^^v#p, art ^jfi^ i.^tt r . 

' r ' ^- . ' ?4 Roflcrlfsii^ "of 7ft*gra^5 h^d rHu V ■ 1 " - ■■ ^ 

\ / dauhE. fcK^lsEs AS to vhcthSF vhe Agency .jniyr^r, 

; Horc^vfr, the ai^ney'r itindirdi of ^thicjl pf;ic^L 

v *\ V irt .net M^^^ .in »uch srtas as nandincr i?tin4i i 



.1 \ ,n.>;^. 



r^n %.,= v= v:j ic a r.,- ^^^^^ ^ - . ^ ■ = ' 

= ^ ^. > -S^'A^;^? sT-. . -v^. . -i iH.'L.i*.^ .r Lv^. V.>.if * , flow V inl f^>r 

i t;. ^ • - - ^ ? ^ I f — i ^ " ^ v .- ! ' t * i : oi t-i-i r.^k i i : ■\ . 

■%-. L:.-^:v " " ? . . ' ^ - 

it.^n,** An4 the Sjtjie^v 

H In ,s J 4 1 1 i on i 45 r i i i ..I ;fj ? f .3 I ^ t m ^ ^ , ih^ a ^ c y g r a c j a 
I- u i^lo^ra ni^t *uvt .1 pf ii^m&n ^imu^ iefin^d by tha 

U f n^" rif!i i If ^ j.Utfi^^£tani 4nJ pr^^^durai for salens l?\g cH* 
i ffic y ' 4 ni '^^^ b^iir^' s-trstb^ffs .ire prater iha^ in les^ 

I I San , ' , . 

^ U lnyL£4« the pvirtUiMtUA ijf puUtc ftfpresentAEiy^ii on all 

^ dy« n©E ic#* ^ AS*rt<^y - : . ^ E 

' ' Cill^l'. gnecuttvt oirriceE of th» prsghia anil ^ffara 4n ^p^o^tunlty 



-I IhQ %ociuty publishes a i^tatemenc of tthics and. requires 
pr-*;r4Pis tQ m^ke th^if e'chic^l pracclces known. 
P.^^ram practices with respect to nondiscrimination In 
.iJ-Usipng arid emplo>^nient af^ mdnltsryd by the agency, 

Zl A--fpvoJ progratjjH are rtevAlyqE^d at intf^rvnl;^ .^hnrter than 
normal cycle under special c 1 r c us g t a nc e s . 

J4 ifii 5-n:nt Un about the Society's stanrf^rdi* development 
pr^cesa is provided in its petition, along with a 

:b descri^^tUn of the methods it employs in evaluating the 
vjlldtty and rgllabiliiy of tho^e standarda. 

-'J Stiiijjfds iiiive been revised, 

I' The Su cUty aUo recently required programi to father and 

g'^b-.it Ln! Qv^^.M ion on program sra.lyatcs. 



C. WHAT AuDltlONAL F ^.C : - A I I N r.^R M AT ^ ON ABOi;? THIS 
^ AGENCY WC^ULD YOU HAV£ Fwi O l^^EFuL IK MAKING 
YOUR DeciSfOK? 
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AfT^R KIXj\\\': the DEJtRlFTtON Of THE 
iiVl^j rHETICAL ACSNCY lELOW, INDLCATE 
YOUR lUDuMSSr OF THE^ NUHEER OF lEARS 
THE HJENCY iHOULD BE RECOGNIZED. 

Circle onj^ of che S ilS^rnaELves co £hf fi^hc: 



2 YRS. 



1 YR. 



DEMY 

RECCn^ilTIOH (0). 



EACH NUMI^ER IN tliE 
COLUMN ro iHE LEFT 

IF r:iE :?e^cr:?ti^n 

CO^SETPOr^DS TO UNI 

■ : i/i Aw i iR i 5 r ic OF 
rn^ livPorHETiCAL 

AHKNCY . 

CtfwLe no ^or€ ^han 
in A i^ovt. 



CI accrediE^ nan-pf,o£it p ,3 s c ^ eu o nd a r y Inacitutloni of all ktnds 

wlchtnlcgrGglsn. 
02 RSAA, whii^h hai been aeClvt tor abouE 20 ye^irs, 
-)i La jppLyLni tor reni^waL of resognition. 
C4 HSAA is A ngn-profLt or ianlzac Ion ; 

^5 vho^i lOla ^unccion ts i c C r ed i c a c I o n . 

. , , ' 

16 Thl-i 4^0 hey Appears to hava vtdgquaCe iCaff to carry ouc Lts 

ac t tvl i t s g p and ^ ' ^ 

07 Che rt^pdn.^ ib i L tc le 3 of Its scaff T^embgra ar# fully daicrlbed 
InChopaiittofi, 



■'S M^ribirs g f a p p r □ p r t a t a proftislonal flalds have a relt in ^he 

a^c r £ t a t Ion proee^^i 
C9 A broad sp^cErum of eduaators Is included on thm aiency's 

pol Id y /d eq ton-mak tni body, 

10 A claar acacamenC of Ehe ieLtcclon procgdurei for th# ilte= , 
ceaffl ^embers Is provided, 

11 with faculcy oi^^.beri and agancy staff making up ch€ site* 
vljttceams, \ ^ ■ 

11 Ln addlcign, the agency ^,alncatns a list of sutitda aoniultan£s. 
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U RSAA hag cUarly eitabltshtd p r s c t d u r as wh 1 e h iniura Imparttal ' 
ev^luattoni and -judgaents* ^ 

14 For examplti mernbefs of thi pol icy /d ec li lon-paklng body do 

noE particfpate in dlscuisLons or dgelgioni affecting IniEiEu- 
cions vUh which chay^re afflilated. 

15 Alio, Che agency glvas 1 ns 1 1 tu £ ioni C hs oppercunity eq IdenElfy 
er^rars of facE or in Etr p r e Ea t io n in Ehe viaiEing teama,* rapore 
andEo^ubinLciuppL«r,€nEaLsaEaflal3* 

U If an accrediced 1 ns c i c^Dion should rafust to allow raevalua- 
tlon, agaFicy policy stlpulit«s chat accridltatlon be viEhdrawn* 

17 Wl^h regard eo coapUlnEi Uvalad ac ins E i Eu E ions , the agaiicy. 
considers "only subs Eant ial ly suppor ted allagatlons which 
could Ifflpalr 'Instlcutional quality or ef f ec t Ivtntii • 
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IS FroGcdurei for deterrnlning p r ea c c red i t a t Ion status are cltarly 
d e ^ c r I b □ d , ^ 

19 and Che proecdurti for abtainlng p r ea c c r ed 1 t a E lo n are appro- 
prtaCQly rolaced to those for full accreditation. 

30 rhg agency'i standards^ which havi recencly been revlBed, 

21 are regularly reviewed by inviting iuggestions from various 
tnieresCed gr o u p g , , 

22 One of the agency's r eq u i r em en c S is Chat Institutions gather 
InforTaationenthetrgraduatis* 



C/WHAT ADDITIONAL FACTL'AL INFORHATION ABOUT THIS 
'^ ACl'iCY WOULD VOU HAVE FQirSO USEFUL IN MAKING 
V0L:R DXCISIUN?/ 

Pliasp Indicate berov. 



1 
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AKTER XfADINa THE □ E SC R C ? T ! 0 *r 0 F THE . ' ' - 

.VJLR Ji;Di,5tiNr JF niE NLj'tSER OF VEARS ■ ' ' 

rHE A;JENCV SHJuLD BE R HCO C N 1 2 ED . ^ ' " 

ClfeU rii '^c the S j U^rnjCiives So Ch^ ri^hEi 4 YRS, ^J -VRS. 2 VRS. I YR, HrCOGNtTlON (0) 



Schools uf Cj-Tipucyf ScLentg Adviiafy Council 



LACK NL'MSER IS THE 
C.3LUMN rO r!{E LEFT 
OF rHE DEiCRirrCCN. 

uHARACTOIJTiC -JF 
MIE yvi'.ITHKTLCAL 
. At:£:lGY, 

Ctrcla no -^sre 5han 
itjv^ jf the nu^berfs 

insL.ion^^'i your juJ^- 



01 

05 



09 
10 

Li 

L> 
13 
.^^ 

15 
16 

^ i? 

1% 

19 



acergdtc^ iniClSuClens offering d eg reef -it all levels in 
conputer 'aa tynae i 

20 ysir 3 , 'M\d ^ ^ ^' 

i$ now ipp lying tor renewal o f r oc sgn 1 e Ion , V 

is a non-profit organisation, vhoae 
primary funcEipnl^iCQaceredic 
p rop r t iry ins t true lens * 

The n^eaey' ar leerla ;or the selection of vlslelng Csam 
rr*?rb'efs a r^ clearly 5 tared g " , 

Specifying that tht tearas are snada up of faculEy a^mbtrs. and 
agency a raf E * . 

In additions iha agency malnCains a" list of outsldt cgnsultants, 
and aliP Includeji A broad spac.trum of conitltuents on lea 
pi3 1 icy / i ^c 1 s ion-making body, 

rha agency's staff rispsns lb 11 it les and qualifications 
a r e de sc r Ibad . 

SfTb'AC e 1 1 ht r encourages or requireg inscttuclons to gather 
Inlorsatlon on.Chitr ifaduatts* 

Clearly degcrlbed procedurss for deteralnlni praaccreditatloh 
are provided J and 

cflrerta for p r eaccr id 1 ta t ion are app f apr la ea ly related to 
thdst employ td for full acc r ad ita t ion > 



lon^ 



The agency's standards have been revised, ^nd 

an active pragraa to tmpfoye accreditation sCandardi or 

cvaiuatlve proctdurgs is carrted out* 

Clearly astablishad prscedures ensure Impartial avaluationa = 
and jud|minCS, - ^, ' , " . , 

as well as provide for fair and t Irae ly t r ea t men t ^y the^agendy 
of cosplalnts against InsiifuElons . 

The agencyVi pa 1 1 1 iof^ s ta tea t ha t it considers an iccuaula t ion 

of complaints rather than a single one* * 
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20 kt the evaluation of the vigltHng teAm report, at leaiC one 
of the visiting tQam semberSvli required to be preient, 

21 and in the event of an adverse dec Isien the agency reports 
the specific reasons- to the chief executive officer if the 
Institution and sffers an opportunity cs appeal. 1. 

22 The agency withdraws an 1 n s t i c u 1 1 on ' g accredited status foe 
refusing to perslt r e e va I ua t 4 en . ' . / 
Finally^ ,the agency performs HQ function Inconsistent with 
the exercise of independent Judgment about the educational 
quality of an ins 1 1 tu t io^n . 



C. WhAI AD?:r!0*^vL FACTVAL INrOSMATlON ABOUt'tHIS 
AJLNCV U^H^LLl xri HAVE rOUND LSEFUL lU MAK H/G ' 
V 0 C S D £ C i 5 : 0 N ? ' ' ' 
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